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ABSTRACT 


This  study  seeks  to  address  the  problem  of  how  to  transfer  expertise  between  generations  of 
Naval  intelligence  analysts.  Expertise  refers  to  the  tacit  and  explicit  knowledge  that  senior 
analysts  call  upon  to  perform  intelligence  analysis.  Transfer  refers  to  the  process  of  “passing  on” 
knowledge  from  senior  analysts  to  those  who  will  replace  them.  Transfer  also  refers  to  adaptive 
expertise  where  a  new  generation  of  analysts  apply  their  freshly-learned  knowledge  in  flexible 
and  creative  ways.  Lessons  are  drawn  from  the  literature  on  knowledge  management,  the 
science  of  learning,  human  information  processing,  and  experience  from  selected  knowledge 
preservation  projects  in  government  organizations.  These  lessons  are  used  to  construct  a  model 
of  knowledge  preservation,  building  on  a  simplified  model  of  knowledge  transfer  derived  from 
the  concepts  of  logical  depth  and  exformation.  A  three-pronged  model,  based  on  how  long 
senior  analysts  are  available  to  work  face-to-face  with  their  replacements,  emerges  from  this 
approach.  A  case  is  made  to  adopt  a  story-centered  approach  to  learning.  Ideas  for  putting  this 
model  into  action  at  the  Office  of  Naval  Intelligence  are  presented.  The  practical  issues  of  cost, 
schedule,  and  cultural  acceptance  are  addressed. 
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EXECUTIVE  SUMMARY 


According  to  a  1998  report  on  the  health  of  Scientific  and  Technical  (S&T)  intelligence, 
analysts  at  the  Office  of  Naval  Intelligence  (ONI)  averaged  over  21  years  of  experience.  One 
consequence  of  this  graying  of  the  work  force  is  an  impending  loss  of  knowledge  and  expertise 
as  these  senior  analysts  retire;  the  accumulated  knowledge  residing  in  their  heads  will  walk  out 
the  door  with  them. 

The  problem  facing  ONI  is  to  transfer  as  much  of  this  knowledge  as  practical  from  senior 
analysts  to  their  replacements.  The  aspiring  analysts  on  the  learning  end  of  this  transaction  have 
different  educational  backgrounds  and  work  experiences.  Since  people  tend  to  learn  most 
effectively  by  building  on  existing  knowledge  and  experiences,  proposed  knowledge 
preservation  solutions  need  to  take  these  individual  differences  into  account.  These  analysts 
begin  their  learning  journeys  at  different  starting  points.  The  senior  analysts  on  the  transmitting 
end  of  this  transaction  will  not  retire  on  the  same  day;  they  will  be  on  hand  to  pass  their 
knowledge  on  for  different  periods  of  time.  Proposed  solutions  need  to  take  advantage  of  the 
time  available. 

The  three-pronged  knowledge  preservation  model  presented  in  this  study  takes  into 
account  the  starting  point  of  analysts  learning  their  craft  and  the  time  available  for  senior 
analysts  to  impart  their  knowledge.  For  senior  analysts  with  more  than  three  years  before 
retirement,  traditional  mentoring  is  the  preferred  approach,  since  there  is  no  substitute  for  the 
one-on-one  interaction  between  mentor  and  mentee.  For  senior  analysts  with  one  to  three  years 
before  retirement,  a  “hand-off’  method  is  the  preferred  approach  since  there  is  insufficient  time 
for  traditional  mentoring  but  enough  time  for  deep  and  substantive  interaction  between  retiring 
experts  and  learners.  In  the  hand-off  method,  learners  build  and  explain  case  studies  based  on 
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senior  analysts’  knowledge  and  experiences.  This  process  encourages  newer  analysts  to  learn 
with  understanding  since  they  will  have  to  explain  and  teach  the  lessons  of  the  case  study.  For 
senior  analysts  with  less  than  a  year  before  retirement,  an  “over-the-wall”  method  is  the 
preferred  approach  since  there  is  insufficient  time  for  direct  human  interaction  but  important 
knowledge  can  still  be  captured  in  a  form  that  fosters  learning  with  understanding.  In  the  over- 
the-wall  method,  a  digital  video  archive  of  stories  is  created  that  can  be  searched  and  viewed 
from  a  learner’s  desktop. 

These  three  learning  strategies  overlap  and  can  be  used  independently  or  together.  It  may 
make  sense  for  some  senior  analysts  to  engage  in  all  three  methods  before  they  retire.  Analysts 
are  engaged  in  traditional  mentoring  today  at  ONI.  These  mentoring  relationships  should  be 
encouraged  and  strengthened.  Learner-built  case  studies  are  similar,  in  principle,  to  the 
“vignette”  sessions  where  analysts  explain  foreign  weapon  systems  to  the  admiral.  In  fact,  it  is 
suggested  to  use  vignettes  as  a  forum  for  learners  to  explain  the  case  studies  they  build.  The  use 
of  digital  video  for  knowledge  preservation  will  be  a  new  practice  at  ONI,  but  there  is  plenty  of 
precedence  for  its  use  at  the  National  Weapons  Laboratories. 

Several  ad  hoc  knowledge  preservation  initiatives  are  being  pursued  at  ONI.  These 
initiatives  and  the  inertia  behind  them  can  propel  the  three-pronged  model  described  in  this  study 
report  into  a  thriving  knowledge  preservation  program.  The  time  to  act  is  now.  Make 
knowledge  preservation  a  priority.  Redirect  the  resources  required  and  invest  in  the  future.  The 
U.S.  Navy’s  future  Naval  intelligence  capability  is  at  stake. 
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SECTION  I 


INTRODUCTION 

An  enemy  cruise  missile  streaks  across  the  sea.  Traveling  several  times  the  speed  of 
sound  and  weaving  in  an  erratic  maimer  it  throws  a  rooster  tail  of  salt  water  thirty  feet  in  the  air 
as  it  passes  within  forty  feet  of  the  wave  caps  below.  Designed  to  penetrate  U.S.  point  defenses 
and  carrying  a  750  lb  warhead,  the  missile’s  large  size  and  hypervelocity  make  its  kinetic  energy 
alone  great  enough  to  devastate  its  intended  target,  a  U.S.  cruiser  seventy  miles  over  the  horizon. 

A  countermeasure  leaps  from  the  sea  quickly  flying  to  intercept  the  missile’s  flight 
trajectory.  In  position  fifteen  miles  in  front  of  the  incoming  missile,  it  waits  six  seconds  before 
detonating  an  explosive  charge  to  create  a  high  power  microwave  beam  aimed  at  the  weaving 
missile,  its  massive  rooster  tail  now  visible  on  the  horizon.  The  microwave  beam  acts  like  a 
shotgun  blast  of  electrical  energy  that  couples  into  the  missile’s  guidance  section  instantly  fusing 
the  attitude  control  circuitry.  The  missile,  still  moving  a  mile  every  second,  slams  into  the  sea 
filling  the  sky  with  a  shower  of  salt  water  and  metal  fragments  from  the  disintegrating  weapon. 

Although  clearly  a  fictional  story,  such  a  scenario  is  plausible  if  its  elements  exist  in  the 
future,  say  the  year  2015.  It  is  a  good  bet  the  enemy  cruise  missile  will  exist,  but  will  the 
analytical  capability  to  devise  countermeasures  to  defeat  it?  Today,  the  U.S.  has  an  edge  against 
potential  adversaries,  in  part,  because  of  the  expertise  of  its  Naval  intelligence  analysts.  In  the 
near  future,  many  of  these  analysts  will  retire  and  their  knowledge  and  expertise  will  leave  with 
them  unless  steps  are  taken  to  capture  and  preserve  their  knowledge. 

In  the  past,  one  generation  of  intelligence  analysts  succeeded  another  without  much  direct 
attention  from  upper  management  and  little  fanfare.  Individual  analysts  entered  the  business  in  a 
“sink  or  swim”  fashion.  They  learned  the  business  through  on-the-job  training.  Some  experts 
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believe  it  takes  6-8  years  of  experience  before  a  new  S&T  analyst  has  the  expertise  necessary  to 
produce  top  notch  analysis.*  During  the  cold  war,  a  steady  stream  of  these  analysts  were 
“moving  up  the  ranks.”  That  picture  has  changed  in  recent  years.  Now  there  are  mostly  newer 
analysts  with  a  few  years  of  experience  and  senior  analysts  with  15  or  more  years  of  experience. 

This  study  report  represents  my  attempt  to  identify  practical  ways  to  transfer  the 
expertise-related  knowledge  of  retiring  Naval  intelligence  analysts  to  their  successors. 

Roadmap  to  the  Study.  I  begin  with  a  discussion  of  an  aging  workforce  and  the 
impending  loss  of  expertise  across  the  Department  of  Defense  and  within  Naval  intelligence. 
This  sets  the  stage  for  providing  the  reader  with  the  motivation  for  pursuing  this  topic  by 
sketching  out  the  dimensions  of  the  problem,  consequences  of  not  addressing  the  problem,  and 
the  inherent  difficulties  of  crafting  a  solution  to  the  problem.  From  here,  the  problem  is  carefully 
stated  to  capture  the  essence  of  the  situation  facing  decision  makers  at  ONI.  An  elegant  solution 
that  solves  the  wrong  problem  is  of  little  value.  After  clearly  defining  the  problem,  the  study 
moves  on  to  consider  lessons  that  can  be  drawn  from  industry,  academia,  and  government. 
These  lessons  touch  on  knowledge  management,  the  science  of  learning,  and  experience  from 
selected  knowledge  preservation  projects  in  government  organizations.  Drawing  on  these 
lessons,  a  knowledge  preservation  model,  based  on  how  long  senior  analysts  are  available  to 
work  face-to-face  with  their  replacements,  is  developed.  The  next  section  considers  ideas  for 
putting  the  model  into  action  at  ONI.  The  practical  issues  of  cost,  schedule,  and  cultural 
acceptance  are  addressed.  Finally,  the  study  ends  with  a  discussion  of  areas  for  future  work  and 
some  concluding  remarks. 

1  William  B.  Scott,  “Intelligence  Complex  Admits  Need  for  Outside  Technical  Talent,”  Aviation 
Week  and  Space  Technology,  April  30,  2001 ,  pp.  75-76. 
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SECTION  II 


MOTIVATION 

DoD  has  been  in  a  sustained  period  of  civilian  workforce  reductions 
since  fiscal  year  1989,  and  additional  reductions  are  expected  at  least 
through  fiscal  year  2005.  The  issue  of  aging  workforce  and  its 
potential  to  adversely  affect  future  mission  accomplishment  is  one  that 
has  been  with  DoD  since  the  beginning  of  its  downsizing,  and  it 
remains  an  issue  today. 

-Michael  Brostek  and  Barry  Holman 
United  States  General  Accounting  Office2 

The  net  effect  is  a  workforce  that  is  not  balanced  by  age  or  experience 
and  that  puts  at  risk  the  orderly  transfer  of  institutional  knowledge. 

Although  we  cannot  say  what  the  appropriate  balance  between  younger 
and  older  employees  should  be,  the  continuing  increase  in  the  number 
of  retirement-age  employees  could  make  it  difficult  for  DoD  to  infuse  its 
workforce  with  new  and  creative  ideas  and  develop  the  skilled  civilian 
workers,  managers,  and  leaders  it  will  need  to  meet  future  mission 
requirements. 

—Henry  L.  Hinton,  Jr. 

United  States  General  Accounting  Office3 

Widespread  Problem.  The  Department  of  Defense  (DoD)  faces  a  crisis  within  its  civilian 
workforce.  The  trends  are  clear.  Insufficient  numbers  of  workers  are  in  the  pipeline  to  fill  the 
shoes  of  their  more-experienced  counterparts  who  will  soon  be  eligible  for  retirement.  Failure 
to  emphasize  long-range  planning  during  past  downsizings,  shifting  demographics,  and 
competition  for  talented  workers  all  contribute  to  the  problem.  Workforce  reductions  were 
accomplished  without  careful  consideration  of  the  skills  and  expertise  that  were  lost  through 
early  retirements,  and  hiring  rates  were  insufficient  to  keep  an  adequate  number  of  successors  in 
the  pipeline.  Demographic  changes  mean  there  are  fewer  workers  available  to  replace  all  the 


2  General  Accounting  Office,  Human  Capital:  Strategic  Approach  Should  Guide  DoD  Civilian 
Workforce  Management  (GAO/T-GGD/NSIAD-00-120,  March  9,  2000),  p.  4. 

3  General  Accounting  Office,  Human  Capital:  Major  Human  Capital  Challenges  at  the 
Departments  of  Defense  and  State  (GAO-01-565T,  March  29,  2001),  p.  8. 
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retiring  baby  boomers.  This  situation  can  be  traced  to  differences  in  birth  rates  between  the 
generations.4 

An  Aging  Workforce  at  Naval  Intelligence.  The  Office  of  Naval  Intelligence  (ONI)  is 

facing  an  impending  loss  of  knowledge  and  expertise  for  the  same  reasons  as  the  rest  of  DoD. 

According  to  a  1998  report  on  the  health  of  Scientific  and  Technical  Intelligence,  analysts  at  ONI 

averaged  over  21  years  of  experience.  The  report  points  out  the  need  to  “capture  remaining 

knowledge  before  it  is  permanently  lost.”5  Some  of  the  challenges  facing  ONI  are  highlighted  in 

the  following  passage  quoted  from  the  report’s  conclusion. 

Analysts  stretched  thin  have  no  time  for  mentoring  the  few  new  S&T 
employees  and  no  time  for  their  own  professional  development 
(assuming  funds  were  available).  The  civilian  analysts,  in  particular,  are 
an  aging  cadre,  many  of  whose  formal  academic  training  was  decades 
ago  and  whose  functional  areas  are  experiencing  exponential  growth.6 

The  problem  of  an  aging  workforce  has  been  recognized  at  ONI  and  across  DoD.  The 
General  Accounting  Office,  the  Office  of  Personnel  Management,  and  DoD  have  performed 
studies  and  initiated  programs  to  combat  its  negative  effects.  Many  of  these  efforts  have 

emphasized  the  need  for  every  agency  to  adopt  structured  workforce  planning  as  a  strategy  to 
align  their  workforce  with  their  future  mission. 

Although  the  next  few  years  will  see  a  loss  of  expertise  and  knowledge  as  older  workers 
retire,  it  also  can  be  a  time  for  DoD’s  civilian  workforce  to  be  infused  with  new  ideas  and  new 


Baby  boomers  have  been  defined  as  those  bom  between  1946  and  1964.  As  an  example 
Susannah  Figura  cites  birth  rates  of  25.3  births  per  1,000  people  in  1954  versus  14.8  in  1973 
See  Susannah  Zak  Figura,  “Leadership  Void:  Experienced  Executives  Are  a  Vanishing  Breed  ” 
Government  Executive,  September  1999.  <http://www.govexec.com/features/>  [April  27  20021 
l“  “/I"  Intelligence  Committee,  “The  Health  of  Scientific  and  Technical 
Intelhgence^  AStudy  Conducted  by  the  Scientific  and  Technical  Intelligence  Committee,”  April 

Ib'd P"  78<http://wwwcia-lc  gov/dci/stic/4253483198/Stic9801-l.html>  [February  5, 2002] 
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blood.  It  is  an  opportunity  for  organizations  to  better  align  their  workforces  with  their  strategic 
plans. 

Workforce  and  Succession  Planning.  According  to  the  Office  of  Personnel 
Management  (OPM),  workforce  planning  is  “the  process  of  ensuring  that  the  right  people  are  in 
the  right  place,  and  at  the  right  time  to  accomplish  the  mission  of  the  agency.  Specifically, 
workforce  planning  is  a  systematic  process  for  identifying  and  addressing  the  gaps  between  the 
workforce  of  today  and  the  human  capital  needs  of  tomorrow.”7 

OPM  created  a  five  step  workforce  planning  model  in  1999  to  help  organizations 
understand  the  process.  Step  one  sets  the  organization’s  strategic  direction.  In  step  two  the 
organization  assesses  its  supply  and  anticipated  demand  for  workers  along  with  discrepancies  in 
needed  skills.  In  step  three,  the  organization  develops  an  action  plan.  Step  four  swings  the  plan 
into  action.  And  in  step  five,  the  organization  monitors,  evaluates,  and  revises  workforce 
planning  to  respond  to  changes  in  the  external  environment  and  the  progress  made  toward 
achieving  goals.8 

Although  sometimes  painful  to  create,  putting  the  mission,  vision,  strategy,  and  strategic 
plan  in  writing  helps  to  clarify  them  and  better  ensure  compatibility  and  linkage  with  existing 
programs.  These  plans  also  provide  a  template  which  can  be  used  to  judge  the  appropriateness  of 
new  initiatives  and  programs.9 


7  Office  of  Personnel  Management,  Workforce  Planning  Desk  Aid  for  Managers  &  Supervisors, 
1999,  <http://www.opm.gov/workforceplanning/>  [April  27, 2002]. 

8  OPM’s  Workforce  Planning  Model  is  available  on  line. 
<http://www.opm.gov/workforceplanning/wQ3model.htm>  [April  27, 2002] 

9  Dina  G.  Levy,  et  al.,  Strategic  and  Performance  Planning  for  the  Office  of  the  Chancellor  for 
Education  and  Professional  Development  in  the  Department  of  Defense  (Santa  Monica,  CA:  The 
RAND  Corporation,  2001),  pp.  7-21. 
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Succession  planning  is  a  form  of  workforce  planning  where  leaders  forecast  the  mix  of 
workers  and  skills  that  will  be  needed  in  the  future  and  then  create  a  plan  to  achieve  the  desired 
future  workforce.  Traditional  succession  planning  efforts  start  with  a  supply  analysis  that  looks 
at  the  existing  workforce  and  competencies  and  a  demand  analysis  that  looks  at  projected  needs, 
staffing  patterns,  and  anticipated  future  workload.  These  products  flow  into  a  gap  analysis  that 
compares  supply  and  demand,  looks  at  demographics,  and  identifies  areas  where  action  needs  to 
be  taken.  Solutions  are  based  on  the  results  of  this  gap  analysis.  Solutions  include  reassignments 
and  recruiting  to  manage  workforce  transition,  employee  training  and  development  to  grow  the 
nght  people,  and  monitoring  staffing  patterns  to  keep  track  of  relevant  changes.10 

Although  planning,  recruitment,  and  retention  all  contribute  toward  achieving  the  desired 
future  workforce,  it  is  the  development  of  existing  and  new  employees  that  commands  our 
anention  here.  How  can  these  workers  learn  the  knowledge  they  will  need  to  solve  tomorrow’s 
problems?  How  can  Are  knowledge  of  senior  workers  nearing  retirement  be  captured,  preserved, 
and  learned  by  workers  who  will  fill  their  shoes? 

Traditional  Career  Development  for  Intelligence  Analysts.  At  ONI,  civilian  intelligence 

analysts  have  traditionally  learned  their  craft  through  “on-the-job”  training  where  they  work  on  a 

number  of  projects  over  several  years.  Not  everyone  is  cut  out  for  intelligence  work.  Some 

people  develop  a  passion  for  the  business  and  make  a  career  out  of  it  while  others  lose  interest 
and  move  on  after  a  few  years. 


AfS16?1  Zl™nermaf  *  “After  You’re  Gone:  Succession  Planning,”  Presentation  at  the 
20m!“rS“ca  Nati°nal  Atainistrative  Conference  2001,  April  23-26, 

<http://www.aftn.ars.usda.gov/AOConf2001/Files/SuccessionPlanning.ppt>  [April  30, 2002] 
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Although  a  new  analyst  is  typically  paired  with  a  more  experienced  analyst  and  assigned 
a  project  to  work  on,  the  process  has  been  described  as  “sink  or  swim.”  It  can  take  up  to  a  year 
for  a  new  analyst  to  begin  to  get  a  good  “feel”  for  the  work  and  be  in  a  position  to  contribute  in  a 
meaningful  way,  for  instance,  to  the  discussion  in  a  meeting.  After  starting  as  an  intelligence 
analyst,  it  typically  takes  a  total  of  6-8  years  before  a  new  S&T  analyst  has  the  expertise 
necessary  to  produce  top-notch  analysis.11 

The  development  process  is  made  more  difficult  because  senior  analysts  are  in  such  short 
supply  and  high  demand.  There  is  insufficient  time  to  spend  coaching  and  mentoring  younger 
analysts  who  are  just  learning  the  craft. 

Time  and  Resource  Constraints.  Just  as  the  Year  2000  or  Y2K  problem  was  impossible 
to  postpone,  so  is  the  aging  workforce  problem.  Every  day  that  passes  brings  these  senior 
analysts  one  day  closer  to  retirement.  Not  only  are  their  days  numbered,  a  large  fraction  of  their 
time  left  will  be  spent  solving  urgent  problems  rather  than  developing  a  new  generation  of 
analysts  to  replace  them.  There  is  a  limited  supply  of  time  and  money  available  for  knowledge 
preservation. 

Importance  of  Naval  Intelligence.  Naval  intelligence  has  been  crucial  in  achieving 
decisive  combat  results.  The  U.S.  Navy’s  expertise  in  Scientific  and  Technical  intelligence  has 
provided  an  edge  through  understanding  how  enemy  weapon  systems  work  and  how  they  can  be 
stopped.  This  capability  is  a  leverage  point  which  can  spell  the  difference  between  victory  and 
defeat. 

Summary.  The  problem  of  an  aging  workforce  exists  across  DoD  and  at  ONI. 
Workforce  and  succession  planning  provide  a  systematic  approach  to  address  this  issue  at  the 

11  ONI  representatives,  Interview  by  author,  Suitland,  MD,  3  April  2002. 
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agency  and  organizational  levels.  Solutions  depend  on  new  workers  to  replace  those  who  will  be 
retiring.  These  retiring  workers  have  a  huge  store  of  knowledge  and  expertise  that  must  be 
passed  on  to  the  next  generation.  There  is  a  limited  supply  of  time  and  money  available  for 
knowledge  preservation.  Some  areas,  like  Naval  intelligence,  are  vital  to  our  national  interests 
and  deserve  special  attention. 
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SECTION  III 


PROBLEM  STATEMENT 

Faced  with  the  increased  complexity  and  proliferation  of  modern 
weapons,  intelligence  organizations  will  be  able  to  function  efficiently 
in  the  future  only  by  making  all  the  necessary  preparations  in  peace¬ 
time.  The  key  to  victory  will  thus  depend  to  a  large  extent  on  the 
quality  of  scientific  intelligence,  an  area  in  which  the  Western 
Democracies  may  have  a  considerable  advantage  if  the  appropriate 
preparations  are  completed. 

—Michael  I.  Handel12 

Given  the  motivation  provided  by  an  aging  workforce  and  the  resource  and  time 
constraints  at  ONI,  is  there  a  practical  way  to  preserve  the  expertise  of  senior  Naval  intelligence 
analysts  before  most  of  them  retire  in  the  next  few  years?  A  solution  to  the  problem  will 
revolve  around  people — senior  analysts  with  knowledge  and  habits  locked  in  their  heads  and 
newer  analysts  tasked  with  learning  old  and  new  tricks.  “It’s  a  people-driven  business,”  said 
Ken  Parys,  ONI’s  Assistant  Director  of  Intelligence  for  Science  and  Technology.  “Our  strength 
is  people  who  can  see  items  that  look  unrelated  on  the  surface,  yet  have  hidden  relationships  and 
patterns  that  seasoned  analysts  can  recognize,”  he  explained.13 

There  are  at  least  three  fundamental  problems  which  must  be  addressed  to  successfully 
preserve  this  knowledge.  These  are: 

•  Nature  of  expertise.  What  constitutes  expertise  in  conducting  Naval  intelligence  analysis 
and  how  does  one  identify  and  prioritize  expertise  worth  preserving? 

•  Capture,  learning,  and  transfer.  How  can  expertise-related  knowledge  be  captured  in  a 
form  that  fosters  learning  and  transfer? 

•  Practicality.  What  characteristics  do  potential  solutions  need  to  possess  to  be  seen  as 
practical  by  analysts  and  senior  Navy  management? 


12  Michael  I.  Handel,  War,  Strategy,  and  Intelligence  (Totowa,  NJ:  Frank  Cass  &  Company 
Limited,  1989),  p.  174. 

13  Kenneth  J.  Parys,  telephone  conversation  with  author,  14  November  2001. 
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These  problems  are  discussed  in  more  detail  below.  However,  before  leaving  this  list,  two  points 
are  worth  noting.  First,  it  is  taken  as  a  given  that  some  of  the  skills  needed  to  conduct 
intelligence  analysis  can  be  learned  by  methods  other  than  direct  experience.  Second,  this  is  not 
intended  as  an  exhaustive  list.  There  are  a  host  of  other  problems  that  need  to  be  considered. 
For  example,  how  can  these  knowledge  preservation  efforts  be  integrated  into  a  larger, 
enterprise-wide,  knowledge  management  program  at  ONI?  How  can  this  knowledge  be  used  to 
create  automated  analytic  tools?  How  can  this  knowledge  be  used  to  improve  the  overall 
analytic  process?  What  tools  can  best  be  used  to  capture  and  share  expertise-related  knowledge? 
Should  outside  consultants  or  an  in-house  team  be  used?  These  are  all  non-trivial  issues  and 
areas  for  productive  further  study.  However,  the  three  fundamental  issues  listed  above  seem  to 
capture  the  essence  of  the  problem  and  must  be  answered  before  more  complicated  questions  are 
engaged.  These  answers  can  perhaps  shed  light  on  a  host  of  other  issues. 

The  Nature  of  Expertise  in  Naval  Intelligence  Analysis.  Expertise  and  knowledge  are 
slippery  concepts  to  pin  down.  Experts  are  often  at  a  loss  to  explain  the  source  of  their  expertise 
and  describe  how  they  get  their  work  done.  Thomas  Davenport  and  Laurence  Prusak  give  the 
following  working  definition: 

Knowledge  is  a  fluid  mix  of  framed  experience,  values,  contextual 
information,  and  expert  insight  that  provides  a  framework  for  evaluating 
and  incorporating  new  experiences  and  information.  It  originates  and  is 
applied  in  the  minds  of  knowers.  In  organizations,  it  often  becomes 
embedded  not  only  in  documents  of  repositories  but  also  in 
organizational  routines,  processes  and  practices.14 

How  can  this  definition  be  used  to  identify  expertise  in  conducting  intelligence  analysis?  If  this 

definition  can  be  cast  in  terms  of  intelligence  analysis  then  several  benefits  may  be  realized. 

14  Thomas  H.  Davenport  and  Laurence  Prusak,  Working  Knowledge:  How  Organizations 
Manage  What  They  Know  (Boston,  MA:  Harvard  Business  School  Press,  1998),  p.  5. 
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First,  it  will  help  specify  expertise  beyond  a  simple  “we  know  it  when  we  see  it”  level  of 
recognition.  Second,  a  more  precise  definition  will  allow  a  better  match  to  be  made  between 
knowledge  and  learning,  distinguishing  those  aspects  that  are  readily  leamable  through  particular 
techniques.  Third,  a  more  precise  definition  will  help  with  prioritization.  If  one  cannot 
distinguish  A  from  B,  then  it  is  difficult  to  prioritize  A  over  B.  Finally,  if  certain  aspects  of 
expertise  in  Naval  intelligence  are  common  with  those  in  other  fields,  then  it  is  more  likely  that 
techniques  that  have  been  shown  to  be  successful  to  capture,  preserve,  and  pass  on  expertise  in 
those  domains  will  be  successful  in  Naval  intelligence. 

Identifying  and  prioritizing  expertise  worth  preserving.  How  can  one  identify  and 
prioritize  expertise  worth  preserving?  Knowledge  must  be  recognized,  which  relates  back  to  the 
discussion  on  definitions  above.  Once  knowledge  is  recognized  as  expertise-related,  it  must  be 
prioritized  so  the  most  critical  knowledge  can  be  captured  using  the  resources  and  time  available. 
Finally,  there  is  the  issue  of  criteria  for  prioritization.  How  does  one  determine  what  is  most 
important  and  valuable?  Should  priority  be  given  to  knowledge  that  is  easiest  to  pass  on  or  the 
most  difficult  to  learn  without  assistance?  Which  of  these  strategies  should  drive  the  priority? 
Perhaps  some  combination  of  strategies  should  be  used. 

Capturing  expertise-related  knowledge  in  a  form  that  fosters  learning  and  transfer. 
How  can  expertise-related  knowledge  be  captured  in  a  form  that  fosters  learning  and  transfer?  It 
is  useful  to  pull  this  question  apart  and  consider  some  of  the  subtleties  lurking  in  it.  Expertise- 
related  knowledge  refers  to  “conditionalized”  knowledge.  That  is,  knowledge  that  includes 
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details  of  the  contexts  in  which  it  applies.15  For  example,  a  rule  of  thumb  needs  to  be  captured 
with  some  information  on  the  conditions  under  which  to  use  it.  The  learning  here  needs  to  be 
thought  of  as  “learning  with  understanding”  versus  the  kinds  of  learning  that  depend  on 
memorization  of  facts  and  rote  procedures.16  Transfer  can  also  be  used  with  two  different 
meanings.  The  first  is  in  the  sense  of  “passing  on”  knowledge  from  one  person  to  another.  The 
second,  and  equally  important  sense,  is  in  the  notion  of  the  “ability  to  extend  what  has  been 
learned  in  one  context  to  new  contexts.”17  Transfer  is  related  to  what  has  been  called  “adaptive 
expertise,  where  the  expertise  can  be  applied  in  new  ways  and  in  new  contexts  rather  just  in 

routine  situations.  Adaptive  expertise  has  been  likened  to  the  difference  between  an  artist  versus 
a  skilled  craftsman. 

Two  more  concepts  that  are  related  to  learning  with  understanding  and  transfer  are 
metacognition  and  pedagogical  content  knowledge.  Metacognition  refers  to  the  ability  to 
monitor  one  s  current  level  of  understanding  and  decide  when  it  is  not  adequate.18  This  ability  is 
important  for  self-directed  learning  and  problem  solving.  This  aspect  of  knowledge  needs  to  be 
made  part  of  the  capture  and  learning  process.  Pedagogical  content  knowledge  refers  to 
understanding  the  barriers  that  appear  as  stumbling  blocks  to  those  trying  to  acquire  knowledge 
for  the  first  time.19  What  are  the  strategies  for  overcoming  these  barriers?  This  may  be  a  more 
difficult  feature  to  capture  than  metacognition,  since  the  experts  may  have  long  forgotten  the 
barriers  they  encountered  when  they  first  acquired  the  knowledge. 

1 5  John  D.  Bransford,  Ann  L.  Brown,  and  Rodney  R.  Cocking,  eds.,  How  People  Learn:  Brain, 
Mind,  Experience,  and  School  (Washington,  D.C.:  National  Academy  Press,  2000),  p.  43. 

<http.//books.nap.edu/books/0309070368/html/index.html>  [  May  6,  2002] 

Ibid.,  pp.  55-56.  ’  J 

X1  Ibid.,  p.51. 

18  Ibid.,  pp.  47-50. 

19  Ibid.,  p.  45. 
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The  need  for  practical  solutions.  What  features  must  potential  solutions  possess  to  be 
seen  as  practical  by  analysts  and  senior  Navy  management?  This  question  is  about 
considerations  of  use  and  the  criteria  by  which  solutions  will  be  judged.  The  analysts’ 
perspective  is  critical  because  if  there  is  a  mismatch  here,  the  analysts  will  resist  using  the 
solution  or,  perhaps,  not  use  the  solution  at  all.  Senior  Navy  management’s  perspective  is 
critical  because  without  their  support,  solutions  will  never  see  the  light  of  day  or  receive  the 
funding  and  attention  required  for  success. 

Candidate  criteria  from  the  analysts’  perspective  include:  ease  of  use,  perceived  ability  to 
improve  analytical  performance,  time  burden,  and  responsiveness  to  individual  needs.  Candidate 
criteria  from  management’s  perspective  include:  performance,  cost,  and  schedule.  Will  the 
proposed  solution  efficiently  capture  and  transfer  information?  Is  the  solution  affordable?  Can 
the  solution  be  implemented  in  time  to  make  a  difference? 

Questioning  the  need  for  knowledge  preservation.  Critics  may  question  the  need  for 
knowledge  preservation.  “Why  expend  resources  to  do  this,”  they  ask,  “when  you  can  just  hire 
smart  people  who  can  figure  out  how  to  conduct  good  analysis?”  After  all,  nobody  taught 
experts  like  R.  V.  Jones  how  to  conduct  analysis.20  And  analysts  develop  their  own  styles.  So 
what  makes  you  think  that  any  knowledge  you  can  capture  will:  a)  be  useful,  and  b)  if  useful,  be 
absorbed  and  used  by  others?  There  is  some  truth  in  all  of  these  points,  but  in  the  final  analysis, 
they  are  moot.  The  following  arguments  explain  why. 


20  R.  V.  Jones  has  been  called  the  father  of  British  scientific  intelligence  during  World  Wax  II. 
His  exploits  in  unraveling  the  mysteries  of  German  radar,  electronic  navigation  aids,  night 
fighters,  and  rocket  programs  serve  as  object  lessons  on  the  value  of  intelligence  analysis.  See 
R.V.  Jones,  Most  Secret  War  (London,  UK:  Hamish  Hamilton  Ltd,  1978). 
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One  option  is  to  hire  smart  people  to  replace  those  retiring  and  let  them  “figure  it  out.” 
However,  these  people  are  sure  to  “re-invent  a  lot  of  wheels”  and  make  mistakes  that  could  be 
avoided  along  their  self-directed  journeys  to  analytical  mastery.  Can  we  afford  the  time  and  loss 
of  capability  while  a  new  generation  learns  their  craft  without  the  benefit  of  knowledge  from 
those  who  went  before  them?  It  is  true  that  nobody  taught  R.  V.  Jones  the  craft  of  intelligence 
analysis.  In  fact,  he  is  credited  with  inventing  the  field  of  scientific  and  technical  intelligence. 
But  R.  V.  Jones  was  a  very  gifted  individual,  at  the  right  place,  and  at  the  right  time  to  make  a 
difference.  Can  the  U.S.  afford  to  pin  its  intelligence  capabilities  on  the  good  fortune  of  having  a 
genius  appear  in  the  nick  of  time? 

While  it  is  true  that  analysts  develop  their  own  styles,  there  are  certain  characteristics  of 
human  psychology  and  analytical  tradecraft  that  are  common  to  all  analysts.21  Analysts  can 
benefit  from  a  better  understanding  of  these  characteristics  without  sacrificing  their  individuality. 
Additionally,  the  expertise  required  to  recognize  patterns  and  draw  relationships  between 
seemingly  disparate  pieces  of  information  is  a  specialty  that  takes  years  to  develop.  If  analysts 
learning  their  craft  can  be  led  away  from  blind  alleys  and  dead  ends  toward  productive  lines  of 
inquiry,  by  developing  effective  techniques  for  organizing  information,  and  through  skill¬ 
building  exercises,  it  stands  to  reason  they  will  have  an  easier  time  developing  to  their  full 
potential. 

Summary.  In  this  section,  three  fundamental  problems  facing  those  who  would  preserve 
the  knowledge  of  retiring  intelligence  analysts  were  described.  They  are:  identifying  and 

21  For  example  see  Richards  J.  Heuer,  Jr.,  Psychology  of  Intelligence  Analysis  (Springfield,  VA: 
National  Technical  Information  Service,  1999) 
<http://www.cia.gov/csi/books/19104/index.html>  [November  12, 2001] 
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prioritizing  expertise;  capturing,  learning,  and  transfer  of  expertise;  and  determining  features  of 
practical  solutions.  Arguments  challenging  the  need  for  knowledge  preservation  were  also 
briefly  considered.  The  next  section  examines  some  lessons  from  industry,  science,  and 
government  that  apply  to  this  problem.  Insights  are  sought  by  looking  at  these  problems  from 
two  different  perspectives:  those  of  knowledge  management  and  the  science  of  learning.  This 
discussion  is  followed  by  a  look  at  some  knowledge  preservation  projects  at  the  National 
Weapons  Laboratories  and  at  the  Naval  Air  Systems  Command. 


15 


SECTION  IV 


LESSONS  FROM  INDUSTRY,  SCIENCE,  AND  GOVERNMENT 

The  belief  that  the  goals  of  understanding  and  use  are  inherently  in 
conflict,  and  that  the  categories  of  basic  and  applied  research  are 
necessarily  separate,  is  itself  in  tension  with  the  actual  experience  of 
science. 

--Donald  E.  Stokes 
Pasteur’s  Quadrant 12 

This  section  begins  with  a  look  at  the  field  of  knowledge  management  which  has  had  its 
widest  application  in  industry  and  academia.  The  academics,  mostly  from  business  management 
schools,  have  concentrated  on  organizational  learning  and  how  companies  can  harness  their 
intellectual  capital  in  the  form  of  employee  knowledge  to  create  new  products  and  services.  A 
theory  of  knowledge  creation  emerges  from  this  body  of  work.  This  will  provide  a  useful 
construct  that  will  contribute  to  the  construction  of  a  model  for  knowledge  preservation. 

The  lessons  from  industry  come  in  the  form  of  knowledge  management  guidelines  the 
academics  have  compiled  from  studying  dozens  of  projects  in  many  different  types  of  firnis. 
These  guidelines  can  be  thought  of  as  “distilled  experience.”  While  the  guidelines  are  somewhat 
generic,  they  do  provide  a  kind  of  touchstone  from  which  to  judge  ideas  for  solutions  to  solve 
problems  created  by  an  aging  workforce. 

Next,  results  are  summarized  from  the  science  of  learning  and  the  related  discipline  of 
cognitive  science  that  bear  on  transferring  knowledge  between  people.  Here  the  message  centers 
around  learning  with  understanding.  This  includes  structuring  knowledge  presentation  so  that 
learners  can  build  on  their  pre-existing  knowledge,  understand  the  contexts  where  new 


22 

Donald  E.  Stokes,  Pasteur  s  Quadrant:  Basic  Science  and  Technological  Innovation 
(Washington,  DC:  Brookings  Institution  Press,  1997),  p.  12. 
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knowledge  applies,  organize  their  knowledge  for  efficient  retrieval  and  application,  and  monitor 
the  state  of  their  own  learning.  These  features  should  drive  the  knowledge  elicitation  process  to 
capture  the  kinds  of  information  that  help  people  leam  efficiently  along  with  the  factual 
knowledge  that  is  required  to  understand  a  subject. 

This  discussion  is  followed  by  a  brief  look  at  some  knowledge  preservation  projects  at 
the  U.S.  Department  of  Energy  (DOE)  and  one  project  at  the  Naval  Air  Systems  Command. 
DOE’s  projects  are  focused  on  preserving  the  knowledge  base  needed  to  operate  a  downsized 
nuclear  weapons  complex.  These  efforts  are  worth  consideration  because  of  similarities  between 
conditions  there  and  at  ONI.  Both  organizations  are  facing  an  aging  workforce  and  both  deal  in 
a  highly  specialized  and  classified  type  of  work.  But  there  are  differences  as  well.  The  U.S.  is 
not  designing  and  building  nuclear  weapons  today,  but  Naval  intelligence  production  needs  to 
continue  uninterrupted.  The  weapons  labs  are  able  to  integrate  their  knowledge  preservation 
efforts  under  a  larger  umbrella  of  Stockpile  Stewardship,  which  has  congressional  momentum 
and  funding,  while  ONI  lacks  the  auspices  of  a  similar  program  for  its  knowledge  preservation 
efforts.  Both  similarities  and  differences  will  be  discussed. 

The  Naval  Air  Systems  Command  has  a  nascent  knowledge  preservation  program.  A 
pilot  program  for  FY03  is  under  consideration.  So  far  results  have  been  mixed.  The  difficulties 
knowledge  workers  have  experienced  in  their  limited  experience  will  be  discussed.  Here  the 
importance  of  pre-planning  and  using  the  right  tools  will  be  emphasized. 

Finally,  an  attempt  will  be  made  to  distill  from  these  lessons  the  critical  underpinnings  of 
a  knowledge  preservation  model  for  DOD  civilian  organizations  in  general  and  Naval 
intelligence  in  particular. 
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Knowledge  Management.  Knowledge  management  is  an  active  area  with  a  growing 
body  of  literature  and  research.  Many  companies  are  making  knowledge  management  a  standard 
practice  to  increase  worker  productivity  and  mitigate  a  shortage  of  knowledge  workers. 

Researchers  are  working  to  study  companies’  practices  to  gain  insights  about  why  projects  fail  or 
succeed.23 


Theory  of  Organizational  Knowledge  Creation.  Ikujiro  Nonaka  and  Hirotaka  Takeuchi 
present  a  theory  of  organizational  knowledge  creation  in  their  seminal  book  on  knowledge 

management.  Their  model  is  based  on  observations  of  activities  in  several  Japanese  firms  over 
many  years.24 

The  gist  of  their  theory  can  be  understood  by  considering  the  two  dimensions  of 
knowledge  creation  and  the  four  modes  of  knowledge  conversion.  Figure  1  shows  the  two 
dimensions  of  knowledge  creation.  The  epistemological  dimension  relates  to  “the  nature  and 
grounds  of  knowledge  especially  with  reference  to  its  limits  and  validity.”25  Here  knowledge 
falls  between  two  extremes.  On  the  one  hand  is  explicit  knowledge  that  can  be  written  down  and 
captured  in  documents  in  the  form  of  words,  diagrams,  and  procedures.  On  the  other  hand  is 
tacit  knowledge  that  is  difficult  to  articulate  into  words  and  is  often  illustrated  with  examples  of 
trying  to  describe  how  to  ride  a  bicycle,  or  how  to  swim,  or  even  how  to  recognize  a  person’s 
face.  “We  know  a  person’s  face,  and  can  recognize  it  among  a  thousand,  indeed  among  a 
million.  Yet  we  usually  cannot  tell  how  we  recognize  a  face  we  know,”  said  Michael  Polanyi  to 


23 

Claire  Mclnemey,  “Knowledge  Management  -  A  Practice  Still  Defining  Itself,”  Bulletin  of  the 
American  Society  for  Information  Science  and  Technology,  February/March  2002,  pp  14-15 
Ikujiro  Nonaka  and  Hirotaka  Takeuchi,  The  Knowledge  Creating  Company:  How  Japanese 
56  94**  Create  theL>ynamics  of  Innovation  (New  York,  NY:  Oxford  University  Press,  1995), 

J^ster’s  Nlflth  New  Collegiate  Dictionary  (Springfield,  MA:  Merriam-Webster,  Inc.,  1985), 


18 


illustrate  tacit  knowledge  via  face  recognition.  “We  can  know  more  than  we  can  tell,”  he 


Figure  1 

Two  dimensions  of  knowledge  creation 
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SOURCE:  Adapted  from  Nonaka  &  Takeuchi  (1995),  p.  57. 


observed.26 

The  ontological  dimension  relates  to  the  existence  or  kinds  of  levels  where  knowledge 
can  be  found.27  Although  “knowledge  originates  and  is  applied  in  the  minds  of  knowers,”28  it  can 
be  observed  at  the  individual,  group,  organization,  and  inter-organization  levels.  As  the  number 
of  people  interacting  increases,  the  difficulty  in  communicating  over  space  and  time  tends  to 
drive  the  distinctions  along  this  dimension  of  knowledge  creation. 


26  Michael  Polanyi,  The  Tacit  Dimension  (Garden  City,  NY :  Doubleday  &  Company,  Inc., 
1966),  p.  4. 

27  Webster’s, p.  825. 

28  Davenport  and  Prusak,  Working  Knowledge,  p.  5. 
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Nonaka  and  Takeuchi  hold  that  the  key  to  knowledge  creation  is  “the  mobilization  and 
conversion  of  tacit  knowledge”  and  a  spiral  that  takes  place  between  the  epistemological  and 
ontological  dimensions.29  Figure  2  depicts  a  spiral  of  knowledge  creation  that  is  projected 


Figure  2 

Knowledge  Creation  Spiral 


forward  in  time  as  knowledge  is  converted  from  explicit  to  tacit  and  back  again  as  it  moves  from 
individuals  to  groups,  organization,  and  between  organizations  and  back  again.  Nonaka  and 
Takeuchi  identify  four  modes  of  knowledge  conversion:  socialization,  extemalization, 
combination,  and  internalization.  Their  model  of  knowledge  creation  is  “anchored  to  a  critical 
assumption  that  human  knowledge  is  created  and  expanded  through  social  interaction  between 


29  Nonaka  and  Takeuchi,  The  Knowledge  Creating  Company,  pp.  56-57. 
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tacit  knowledge  and  explicit  knowledge.”30  It  is  useful  to  examine  these  modes  since  they 
appear  again  in  the  discussion  on  the  science  of  learning  and  play  a  role  in  shaping  the 
knowledge  preservation  model  introduced  in  the  next  section. 

The  four  modes  have  names  that  help  to  generalize  many  specific  examples  into  a  few 
categories.  Figure  3  illustrates  the  four  modes  of  knowledge  conversion  and  their  resulting 
contents.  These  modes  can  also  be  understood  in  terms  of  everyday  examples. 
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SOURCE:  Adapted  from  Nonaka  &  Takeuchi  (1995),  pp.  71-72. 

Tacit  knowledge  is  converted  to  tacit  knowledge  between  people  in  the  form  of  shared 
experiences.  This  conversion  process  is  called  socialization.  The  contents  of  this  knowledge  can 
be  characterized  as  “ sympathized ”  knowledge  since  the  process  implies  a  connection  between 
people  that  transcends  written  and  spoken  language.  To  understand  this  mode  of  knowledge 
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Knowledge  Conversion  Modes  and  Contents 
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conversion,  consider  wha,  happens  when  two  peopie  connnnnicare  who  have  ridden  the  same 
roller  coasier  a,  an  amusement  park.  When  they  talk  to  each  other  about  the  feeling  of  helpless 
anticipahon  one  experiences  near  rite  end  of  rite  long  climb  to  rite  first  big  drop,  they  can 
instantly  relate  to  each  other.  Both  have  vivid  m^nories  of  ft*  experiences.  When  they  talk  to 

unable  to  share  this  knowledge. 

Tacit  knowledge  is  converted  to  explicit  .mow, edge  in  the  fomr  of  metaphors  and 
analogies.  This  process  can  occur  within  a  single  person  or  in  a  larger  group.  This  type  of 
knowledge  conversion  is  called  Nation.  As  implied  by  rite  use  of  metaphor  and  analogy 
•he  contents  are  concept  and  make  use  of  knowledge  people  already  possess  to  ge,  a  point 
across.  Continuing  wifi,  the  roller  coaster  example,  a  person  with  roller  coaster  experience  may 
be  able  to  convey  a  sense  of  helpless  anticipation  one  feels  approaching  rite  first  big  drop  by 
towmg  on  a  metaphor  of  being  simultaneously  mapped  in  an  elevator  while  anticipating  rite 
feeling  you  ge,  when  rite  pi,  of  your  stomach  drops  as  in  an  aitplane  in  rough  ah.  Although  i,  is 

experience  people  have  of  feeling  helpless  and  most  people  have  experienced  rite  pi,  of  riteir 
stomach  dropping  either  in  an  aitplane  flying  in  rough  weather  or  on  as  a  child  on  a  swing  set. 
Again  the  emphasis  is  building  on  wha,  someone  already  knows. 

Tacit  knowledge  is  created  from  explicit  knowledge  when  people  “internalize” 
knowledge  either  by  being  able  explain  “i,  in  their  own  tetms”  or  practicing  a  ski,,  until  i, 
becomes  second  nature.  This  type  of  knowledge  conversion  is  called  internalization.  The 
contents  of  this  conversion  ptocess  can  be  characterized  as  operational  knowledge,  parricularly 
when  people  am  learning  a  ski,,  or  operational  procedure.  This  convemion  process  can  be  seen 


as  learning  at  the  individual  level.  It  also  depends  on  building  on  what  people  already  know.  An 
example  here  would  be  the  roller  coaster  operator  learning  to  operate  the  ride  by  reading 
manuals,  diagrams,  and  stories  about  potential  problems  like  how  to  handle  the  ride  on  days  with 
peak  crowds. 

Explicit  knowledge  is  created  from  explicit  knowledge  when  people  combine  knowledge 
from  several  sources.  Accordingly,  this  type  of  knowledge  conversion  is  called  combination. 
The  contents  from  this  conversion  process  can  be  thought  of  as  systematic  knowledge  and  the 
artifacts  or  products  can  be  in  any  form  of  explicit  knowledge.  Collecting  the  research  to  write 
this  study  report  is  an  example  of  combination.  A  roller  coaster  related  example,  might  have  a 
design  engineer  combining  design  features  from  several  rides  to  design  a  new  coaster. 

Nonaka  and  Takeuchi  draw  from  several  examples  of  innovation  in  Japanese  companies 

O  1 

to  defend  and  illustrate  their  theory.  Each  of  these  four  conversion  processes  in  knowledge 
creation  can  provide  insights  into  how  to  leverage  a  learner’s  existing  knowledge  and  why 
human  interaction  is  important. 

Nonaka  and  Takeuchi  identify  several  enabling  conditions  for  organizational  knowledge 
creation.  These  enabling  conditions  include  intention,  autonomy,  fluctuation  and  creative  chaos, 
redundancy,  and  requisite  variety.  Nonaka  defines  organizational  intention  as  an  “organization’s 
aspirations  toward  its  goals”32  Intention  also  provides  a  way  for  people  to  judge  the  value  of 
information  and  knowledge. 

Autonomy  allows  individuals  to  act  as  circumstances  dictate.  This  freedom  fosters 
creativity  and  lets  people  take  responsibility  for  their  day  to  day  activities.  Autonomy  applies  to 


31  Ibid.,  pp.  95-123. 
^  Ibid.,  p.  75. 
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^oups  as  well  and  helps  to  explain  why  companies  sometimes  use  a  ‘skunk  works’  approach  to 
create  new  products  and  ideas. 


FlUaUa,i°”  and  ma,tVe  ckaos  “  because  they  ‘-stimulate  the  intera«ion 

between  the  organrzation  and  its  external  environment.”33  Fluctuation  implies  a  back  and  forth 

movement  with  uncertoty.  when  fluctuation  is  induced  into  an  organization,  people’s 
routines  and  habits  can  be  interrupted  creating  an  opportunity  for  sensemaking.  People  question 
their  assumptions  and  take  a  fresh  look  a.  their  environment  and  situation.  Creative  chaos  helps 
to  induce  a  sense  of  urgency  and  serves  to  focus  people’s  attention  on  problem  solving.  Nonaka 
and  Takeuchi  point  out  that  Japanese  companies  sometimes  employ  creative  chaos  through  the 
purposeful  use  of  ambiguity  such  as  ambiguous  vision  statements  from  upper  management. 

However,  they  caution  that  benefits  from  creative  chaos  can  only  occur  when  people  have  the 
opportunity  to  “reflect  on  their  actions.”34 

Redundancy  is  where  the  common  ground  needed  for  people  to  communicate  and  lean, 
from  one  another  originates.  I,  is  the  existence  of  shared  information  that  goes  beyond 
te  project  needs  and  is  seen  m  the  overlapping  of  boundaries  and  responsibilities  in  a 
group.  The  idea  is  that  sharing  “redundant  information  promotes  the  sharing  of  tacit  knowledge, 
because  individuals  can  sense  what  others  are  trying  to  articulate.”35 

The  idea  behind  requisite  variety  is  that  an  organization’s  internal  diversity  must  match 
the  variety  and  complexity  of  its  environment  in  order  to  effectively  respond  to  challenges. 
People  with  requisite  variety  can  deal  with  many  contingencies.  Here  access  to  information  from 
tough  tire  organization  can  help.  Other  techriques  to  foster  requisite  variety  include  creating  a 


33  Ibid.,  p.  78. 

34  Ibid.,  p.  79. 

35  Ibid.,  pp.  81-81. 
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flat  organizational  hierarchy  connected  with  a  network  and  undergoing  frequent  organizational 
restructurings.36 

Now  that  a  theory  for  knowledge  creation  has  been  reviewed,  it  is  time  to  look  at  some 
guidelines  for  successful  knowledge  management  projects.  These  guidelines  are  of  interest 
because  knowledge  preservation  is  one  form  of  knowledge  management. 

Guidelines  for  Knowledge  Management  Projects.  What  are  the  characteristics  of 
successful  knowledge  management  projects?  Based  on  their  study  of  thirty-one  different 
projects  in  twenty  different  firms,  Davenport  et  al.  identified  nine  common  factors  of  successful 
projects.  They  are:37 

•  A  knowledge-oriented  culture 

•  Technical  and  organizational  infrastructure 

•  Senior  management  support 

•  A  link  to  economics  or  industry  value 

•  A  modicum  of  process  orientation 

•  Clarity  of  vision  and  language 

•  Nontrivial  motivation  aids 

•  Some  level  of  knowledge  structure 

•  Multiple  channels  for  knowledge  transfer 

A  Knowledge-Oriented  Culture.  A  knowledge  oriented  culture  is  about  employees’ 
attitudes  toward  sharing  information  and  exploring  new  ideas.  Sometimes  workers  have  barriers 
toward  sharing  knowledge  because  of  fear  of  job  loss  or  loss  of  prestige.  This  can  occur  when 
there  is  competition  for  resources  and  projects  that  could  lead  workers  to  fear  they  will  be  “left 
out”  if  they  share  their  exclusive  knowledge.  The  knowledge  management  project  needs  to  fit 
the  organization’s  culture.  Trying  to  force  one  suite  of  tools  on  workers  who  are  accustomed  to 


36  Ibid.,  pp.  82-83. 

37  Davenport  and  Prusak,  Working  Knowledge,  p.  153.  Also  see  Thomas  H.  Davenport,  David 
W.  De  Long,  and  Michael  C.  Beers,  “Successful  Knowledge  Management  Projects,”  Sloan 
Management  Review,  Winter  1998,  pp.  43-57. 
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being  able  to  choose  their  own  tools  probably  will  not  succeed.  Expectations  and  routines  are 
important  to  develop  a  knowledge  sharing  culture.  They  are  where  the  inertia  resides.  If  people 
expect  to  share  knowledge  and  it  is  a  part  of  their  daily  routines,  then  knowledge  projects  can 
flourish.  Otherwise  expectations  and  routines  have  to  change  for  knowledge  projects  to  succeed. 

Technical  and  Organizational  Infrastructure.  Projects  that  are  able  to  take  advantage 
of  and  leverage  existing  technical  infrastructure  have  an  advantage.  Not  only  do  they  help  defray 
costs  by  using  existing  networks  and  systems,  they  generally  have  fewer  problems  to  resolve 
because  others  have  already  grappled  with  these  problems. 

Senior  Management  Support.  Senior  management  support  is  important  because  they 
set  the  organization's  direction.  They  send  messages  that  make  clear  the  kind  of  knowledge  that 
is  important  to  the  organization.  They  can  also  help  clear  obstacles  and  roadblocks  from  the 
project’s  path.38 

A  Link  to  Economics  or  Industry  Value.  Having  a  link  to  economics  or  industry  value 
almost  goes  without  saying.  The  investment  in  time,  technology,  and  energy  required  for  a 
knowledge  management  project  needs  to  be  important  enough  to  justify  the  effort.  Dollars 
earned  or  saved  are  the  most  tangible  things  to  look  at  in  industry,  but  other  indirect  measures 
such  as  reduced  product  cycle  time  or  increased  customer  satisfaction  can  be  used  as  well.39  For 
intelligence  analysis,  one  might  look  at  increased  analytical  capabilities  and  improved  quality  of 
analysis. 

A  Modicum  of  Process  Orientation.  Davenport  and  Prusak  advocate  taking  a  process 
perspective  to  help  the  project  manager  “have  a  good  sense  of  his  or  her  customer,  the 


™  Ibid.,  p.  156. 
39  Ibid.,  p.  157. 
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customer’s  satisfaction,  and  the  productivity  and  quality  of  the  services  offered.”40  A  process 
orientation  forces  one  to  systematically  pay  attention  to  those  areas  that  might  otherwise  “slip 
through  the  cracks”  with  a  more  ad  hoc  approach. 

Clarity  of  Vision  and  Language.  Clarity  of  vision  and  language  make  it  easier  for 
workers  to  understand  what  the  project  is  about  and  where  the  value  is  for  them  and  the 
organization.  Since  many  knowledge  management  related  terms  have  several  meanings,  it  is 
important  to  pinpoint  the  specifics  and  scope  of  a  project  to  avoid  confusion.  It  is  also  important 
to  exclude  unstructured  data  or  other  information  that  is  not  compatible  with  the  project.41 

Nontrivial  Motivational  Aids.  Nontrivial  motivational  aids  need  to  be  integrated  into 
the  long-term  design  of  a  knowledge  management  project.  “Knowledge,  being  intimately  bound 
up  with  people’s  egos  and  occupations,  does  not  emerge  or  flow  easily.  Employees  must 
therefore  be  motivated  to  create,  share,  and  use  knowledge.”  Davenport  and  Prusak  found 
examples  of  where  short-term  incentives,  airline  frequent-flyer  mileage  for  instance,  were  not 
effective  in  getting  employees  to  use  knowledge  management  systems.  They  emphasize  the  use 
of  long-term  incentives  tied  to  the  performance  appraisal  and  compensation  structure  and  the  use 
of  highly  visible  awards  programs.42 

Some  Level  of  Knowledge  Structure.  Successful  knowledge  management  projects  have 
some  knowledge  structure,  but  not  too  much.  If  the  structure  is  too  fine-grained  and  rigid  then  it 
becomes  difficult  to  accommodate  new  forms  of  knowledge  and  new  categories.  If  there  is  no 
structure,  it  becomes  very  difficult  to  find  the  information  you  are  looking  for,  and  the  system 


40  Ibid.,  p.  157. 

41  Ibid.,  p.  158. 

42  Ibid.,  p.  158. 
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essentially  becomes  unusable.  The  fluid  nature  of  knowledge  requires  that  the  structure  be 
flexible  and  tailored  to  meet  worker’s  patterns  of  use.43 

Multiple  Channels  for  Knowledge  Transfer.  There  are  at  least  two  reasons  why 
multiple  channels  of  knowledge  transfer  make  sense.  First,  different  users  have  different  needs. 
And  in  the  case  of  the  knowledge  preservation  model  introduced  in  the  next  section,  the  experts 
may  only  be  available  for  a  limited  time.  Multiple  channels  offer  one  way  to  make  better  use  of 
that  time.  Second,  multiple  channels  reinforce  one  another.  Here  is  an  example  where  network 
channels  needed  to  be  reinforced  with  face-to-face  meetings: 

Some  of  the  firms  that  had  knowledge  repositories  realized  that  they  had 
to  get  contributors  together  in  a  face-to-face  setting  on  a  regular  basis. 

In  that  “high  bandwidth”  situation,  trust  can  be  established,  structures 
[°Lrknowledge  can  be  developed,  and  difficult  issues  can  be  resolved. 

MIT  researcher  Tom  Allen  has  found  in  many  studies  that  scientists  and 
engineers  exchange  knowledge  in  direct  proportion  to  their  level  of 

persona1  contact  There  is  a  strong  need  for  what  the  U.S.  Army 
calls  face  time. 

Having  discussed  the  characteristics  of  successful  knowledge  management  projects,  what 
are  the  pitfalls  to  avoid?  Fahey  and  Prusak  identified  “the  eleven  deadliest  sins  of  knowledge 

management”  based  on  their  study  of  over  a  hundred  projects  over  a  five-year  period.  The 
pitfalls  they  identified  are:45 

•  Not  developing  a  working  definition  of  knowledge 

•  Emphasizing  knowledge  stock  to  the  detriment  of  knowledge  flow 

•  Viewing  knowledge  as  existing  predominantly  outside  the  heads  of  individuals 

•  Not  understanding  that  a  fundamental  intermediate  purpose  of  managing 
knowledge  is  to  create  shared  context 

•  Paying  little  heed  to  the  role  and  importance  of  tacit  knowledge 

•  Disentangling  knowledge  from  its  uses 


43 

44 

45 


Ibid.,  p.  159. 
Ibid.,  p.  159. 
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•  Downplaying  thinking  and  reasoning 

•  Focusing  on  the  past  and  the  present  and  not  the  future 

•  Failing  to  recognize  the  importance  of  experimentation 

•  Substituting  technological  contact  for  human  interface 

•  Seeking  to  develop  direct  measures  of  knowledge 

Not  Developing  a  Working  Definition  of  Knowledge.  Not  developing  a  working 
definition  of  knowledge  relates  to  the  success  guideline  about  clarity  of  vision  and  language.  By 
not  having  a  good  definition  of  knowledge  in  a  project,  one  is  vulnerable  to  misinterpretation  and 
confusion  stemming  from  ambiguous  terminology.  It  is  important  to  distinguish  data  and 
information  from  knowledge.  Data  and  information  are  usually  not  directly  related  to  personal 
beliefs  and  values  or  “derived  from  minds  at  work,”  while  knowledge  always  originates  at  a 
personal  level.46 

Emphasizing  Knowledge  Stock  to  the  Detriment  of  Knowledge  Flow.  Emphasizing 
knowledge  stock  to  the  detriment  of  knowledge  flow  is  really  about  the  need  to  achieve  a  balance 
in  the  efforts  to  collect  and  store  knowledge  with  efforts  to  disseminate  and  make  knowledge 
accessible.  This  guideline  brings  to  mind  an  image  of  dam  filled  to  capacity  while  crops  in  fields 
downstream  shrivel  and  wilt  due  to  lack  of  water.  Fahey  and  Prusak  describe  how  viewing 
knowledge  in  terms  of  information  can  lead  to  a  “stock-oriented”  mentality.  They  also  point  out 
how  knowledge  is  dynamic  and  alive  in  the  community  of  people  it  originates  in  and  therefore 
view  flow  more  as  a  ongoing  interchange  like  tidal  waters  versus  the  downstream  flow  of  a 
river.47 

Viewing  Knowledge  as  Existing  Predominately  Outside  the  Heads  of  Individuals. 

Viewing  knowledge  as  existing  predominately  outside  the  heads  of  individuals  can  lead  to  two 

46  Davenport  and  Prusak,  Working  Knowledge,  pp.  2-6. 

47  Fahey  and  Prusak,  “The  Eleven  Deadliest  Sins  of  Knowledge  Management,”  pp.  266-67. 
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types  of  problems.  The  first  comes  from  disregarding  the  fact  that  knowledge  originates  in  a 

knower’s  mind  and  derives  its  context,  meaning,  and  value  from  human  use.  Although 

knowledge  can  reside  in  documents  and  be  embedded  in  processes  and  routines,  it  is  inextricably 

linked  with  human  thought  and  action.  The  second  type  of  problem  involves  a  tendency  to 

elevate  data  and  information  beyond  their  limits. 

Yet  organizations  seem  to  view  knowledge  as  if  it  has  a  life  of  its  own. 

They  dub  strikingly  mundane  databases  as  “knowledge  bases,”  they  talk 
of  search  engines  as  if  they  were  human  brains,  and  they  extol  executive 
expert  systems  as  if  the  human  mind  were  incidental  to  their 
construction  and  use.  This  attempt  to  dress  up  decades  old  technologies 
and  concepts  in  new  “knowledge”  clothing  is  one  of  the  more  serious 
distractions  faced  by  knowledge  advocates4® 

Not  Understanding  Shared  Context.  Not  understanding  that  a  fundamental 
intermediate  purpose  of  managing  knowledge  is  to  create  shared  context  gets  at  the  idea  that 
knowledge  requires  a  connection  between  people.  People  have  a  shared  context  when  they 
achieve  a  shared  understanding  of  the  world  and  its  many  connections.  Their  shared 
understanding  is  built  on  common  mental  models  which  result  from  experiences,  reflection,  and 
dialogue.  These  activities  take  time.  Fahey  and  Prusak  view  dialogue  as  “not  just  discussion  but 
the  presentation  and  consideration  of  different  views  and  perspectives  with  the  purpose  of 
developmg  a  distinctly  different  or  new  view  or  perspective.”49  In  other  words,  it  is  the  dialogue 

that  creates  knowledge.  A  static  snapshot,  frozen  in  time,  captured  in  a  knowledge  “stock” 
diminishes  in  value. 

Paying  Little  Heed  to  Tacit  Knowledge.  Paying  little  heed  to  the  role  and  importance 
of  tacit  knowledge  is  a  mistake  that  is  easy  to  make.  Tacit  knowledge  is  hard  to  get  your  arms 
around.  Tacit  knowledge  is,  by  definition,  not  easily  expressed  in  words.  So  much  of  our  tacit 

48  Ibid.,  p.  267. 

49  Ibid.,  p.  268. 
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knowledge  is  taken  for  granted,  it  is  easy  to  lose  sight  of  how  much  we  depend  on  it  to  perform. 
Echoing  a  central  theme  from  Nonaka  and  Takeuchi’s  theory  of  knowledge  creation,  Fahey  and 
Prusak  state  that:  “tacit  knowledge  is  the  means  by  which  explicit  knowledge  is  captured, 
assimilated,  created,  and  disseminated.”50  Fahey  and  Prusak  attribute  the  failure  to  attend  to  tacit 
knowledge  to  project  managers’  lack  of  understanding  and  appreciation  for  the  interaction 
between  tacit  and  explicit  knowledge.  Fearing  that  tacit  knowledge  is  “inaccessible  and 
impossible  to  influence,”  they  often  downplay  and  de-emphasize  the  tacit  dimension.  This  can 
cause  their  efforts  to  achieve  results  strictly  through  explicit  knowledge  to  stall.51 

Disentangling  Knowledge  From  Its  Uses.  Disentangling  knowledge  from  its  uses  is 
another  pitfall  related  to  the  need  for  balance.  Project  managers  guilty  of  committing  this  sin 
spend  too  much  effort  perfecting  the  technical  aspects  of  refining  and  organizing  the  data  and 
information  at  the  expense  of  focusing  on  relevance  and  the  ability  to  influence  decisions  and 
strategic  direction.  They  miss  the  forest  for  the  trees. 

Downplaying  Thinking  and  Reasoning.  Downplaying  thinking  and  reasoning  is  a 
pitfall  intimately  linked  to  an  organization’s  culture.  On  the  one  hand,  Americans  have  been 
raised  in  an  educational  system  where  clear  thinking  and  the  ability  to  reason  effectively  have 
not  been  emphasized.52  This  can  result  in  workers  who  never  learned  to  write  clearly,  and 
therefore  exhibit  muddled  thinking,  flawed  logic,  and  faulty  assumptions  as  they  struggle  to 

50  Ibid.,  p.  268. 

51  Ibid.,  p.  269. 

52 

William  Zinsser  discusses  this  problem  and  one  solution  to  it  based  on  “writing  around  the 
curriculum”  where  learning  to  write  (and  think  clearly)  is  extended  beyond  traditional  English 
classes.  Students  take  specially  designed  courses  in  subjects  from  economics  to  chemistry  where 
writing  and  thinking  about  the  elements  of  good  writing  are  a  central  part  of  the  learning 
experience.  See  William  Zinsser,  Writing  to  Learn:  How  to  Write — and  Think — Clearly  About 
any  Subject  at  All  (New  York,  NY:  Harper  &  Row,  Publishers,  1988),  pp.  42-51. 
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create  knowledge  products.  On  the  other  hand,  deeply  held  beliefs  and  assumptions  can  act  as  a 
barrier  to  keep  managers  from  seeing  signs  of  changing  conditions  and  considering  their  impact. 
Thinking  and  reasoning  are  important  because  people  reach  different  states  of  knowledge 
through  reasoning,  for  example,  in  a  process  that  leads  to  changes  in  a  product’s  design  based  on 
designers  thinking  and  reasoning  about  how  customers  actually  use  the  product.53 

Not  Focusing  on  the  Future.  Focusing  on  the  past  and  the  present  and  not  the  future  is  a 
pitfall  that  stems  in  part  from  the  difficulty  of  projecting  knowledge  into  the  future  and  in  part 
from  a  misunderstanding  of  the  relationship  between  knowledge  and  the  future.  It  is  much  easier 
to  collect  knowledge  and  information  from  the  past  and  present  than  it  is  to  make  judgments 
about  the  future.  Scenario  planning  is  one  technique  that  has  been  used  help  identify  and  frame 
possible  future  situations.  “If  the  intent  of  knowledge  is  to  inform  and  influence  decision 
making,  then  its  focus  must  be  on  the  future.  Although  we  cannot  know  the  future,  every 
strategy,  decision,  and  action  is,  by  definition,  premised  upon  some  view  of  the  future.”54  Most 
knowledge  preservation  projects  have  limited  resources.  To  be  successful,  projects  must  focus 
on  the  future  relevance  of  the  knowledge  being  preserved. 

The  Importance  of  Experimentation.  Failing  to  recognize  the  importance  of 
experimentation  can  handicap  a  project’s  ability  to  create  new  knowledge  since  experiments  are 
often  essential  to  validate  new  ideas  and  explore  new  techniques  for  solving  problems.  Fahey 
and  Prusak  point  out  how  documenting  best  practices  can  sometimes  serve  to  inhibit  future 


54  Fahey  ^  Prusak?  “The  Eleven  Deadliest  Sins  of  Knowledge  Management,”  pp.  270-71. 
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improvements  to  these  practices  (to  avoid  updating  the  documentation)  and  caution  about  the 
tendency  to  emphasize  exploitation  over  exploration  of  new  knowledge.55 

Substituting  Technological  Contact  for  Human  Interface.  Substituting  technological 
contact  for  human  interface  can  be  a  natural  tendency  with  network  technologies  such  as  email 
and  web  servers  and  with  a  workforce  distributed  in  different  locations.  While  these  tools 
provide  convenience  and  access,  they  lack  the  “high  bandwidth”  interaction  of  one-on-one 
contact  between  people  needed  to  create  the  shared  context  so  critical  to  successful  knowledge 
projects.56 

Seeking  to  Develop  Direct  Measures  of  Knowledge.  Finally,  expecting  all  benefits  of 
knowledge  management  to  be  captured  by  a  few,  direct  measures  may  cause  knowledge  efforts 
to  be  canceled  before  they  “pay  off’  or  encourage  managers  to  change  their  focus  to  “make  the 
numbers  look  better.”  For  instance,  instead  of  measures  of  access  “hits”  and  database  statistics, 
the  focus  needs  to  be  on  outcome-based  measures  that  look  at  activities,  consequences  and 
results.57  Of  course,  these  broader  measures  are  difficult  to  design  and  probably  will  still  not 
easily  capture  all  the  project’s  benefits. 

Encouraging  the  nine  factors  of  successful  projects  while  avoiding  the  eleven  pitfalls 
increases  the  likelihood  that  a  knowledge  preservation  project  will  be  successful.  Recognizing 
the  pitfalls  is  one  thing,  learning  how  to  avoid  them  is  another.  However,  these  guidelines  can 
serve  as  a  useful  checklist  from  which  to  judge  the  knowledge  preservation  model  presented  in 
the  next  section. 


55  Ibid.,  p.  272. 

56  Ibid.,  p.  273. 

57  Ibid.,  pp.  273-74. 
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Having  introduced  a  theory  of  knowledge  creation  in  commercial  settings  and  discussed 
guidelines  for  knowledge  management  projects,  the  attention  now  shifts  to  consider  lessons  the 
science  of  learning  has  to  offer  for  crafting  knowledge  preservation  solutions. 

The  Science  of  Learning.  Results  from  cognitive  science  on  how  the  mind  works  are 
helping  to  move  from  learning  practices  based  on  speculation  to  ones  underpinned  by  science. 
Some  of  these  results  offer  lessons  for  knowledge  preservation  efforts  aimed  at  capturing  the 
expertise  of  retiring  experts.  What  it  means  for  one  to  leam  with  understanding  is  a  key  result. 

Learning  with  understanding  implies  more  than  just  being  able  to  recite  a  mass  of 
disconnected  facts  and  solve  test  problems  in  a  classroom  setting.  When  one  understands  a 
subject,  one  is  able  to  size  up  problems  and  frame  them  from  different  vantage  points.  One  is 
also  able  to  think  clearly  enough  about  the  subject  to  explain  it  to  other  people.  Understanding  is 
difficult  to  study  in  people,  so  cognitive  scientists  have  turned  to  the  study  of  experts  and 
expertise  to  gain  insights  into  what  understanding  entails.  Researchers  have  found  that  to  solve 
problems,  experts  depend  on  a  rich  body  of  knowledge  which  must  be  organized  around 
important  concepts,  and  the  knowledge  must  be  “contextualized”  to  specify  the  limits  of  its 
applicability.58 

One  of  the  developments  in  cognitive  psychology  that  has  a  direct  bearing  on  knowledge 
preservation  is  an  increased  understanding  of  how  experts  differ  from  novices.  This  research  has 
shown  that  experts  are  able  to  solve  problems  through  fluent  retrieval  of  pertinent  facts  and  the 


58  * 

Summaries  of  this  research  with  references  to  the  original  sources  include  Bransford  et  al., 
How  People  Learn,  pp.  8-9,  pp.  31-50  and  Daniel  Reisberg,  Cognition:  Exploring  the  Science  of 
the  Mind  2n  Ed.  (New  York,  NY:  W.  W.  Norton  &  Company,  2001),  pp.  466-468. 
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distinctive  ways  in  which  their  knowledge  is  organized.59  A  knowledge  preservation  effort  must 
pay  particular  attention  to  these  details  and  seek  ways  to  elicit  them  to  get  learners  on  the  right 
track  to  develop  similar  skills  and  ways  of  organizing  knowledge. 

Another  area  where  research  results  are  of  interest  deals  with  learning  and  transfer  and 
how  to  structure  learning  experiences  so  people  are  able  to  use  what  they  learn  in  new  settings. 
This  encourages  learners  to  step  back,  look  at  the  big  picture,  and  to  notice  patterns. 
Approaching  an  initial  problem  statement  as  a  point  of  departure  and  an  opportunity  for 
exploration  is  another  trait  learners  should  strive  for.  The  ability  to  monitor  and  evaluate  one’s 
own  current  level  of  understanding  is  also  critical  to  the  flexible  application  of  knowledge  in  new 
situations.60  All  of  these  areas  of  research  relate  to  what  it  takes  for  people  to  learn  with 
understanding. 

Another  finding  of  relevance  to  knowledge  preservation  is  that  the  new  knowledge 
people  leam  must  be  constructed  from  and  combined  with  their  existing  knowledge.  This  means 
that  a  learner’s  existing  knowledge,  beliefs,  and  preconceptions  strongly  influence  the  learning 
process.  The  degree  to  which  this  existing  knowledge  is  engaged  and  challenged  determines  the 
depth  of  understanding  reached  with  the  new  knowledge.61 

Since  understanding  is  important,  learners  need  to  “take  control  of  their  own  learning”  to 
monitor  when  they  understand  a  subject  and  when  they  need  more  material  and  practice.  This 

59  Bransford,  et  al.,  How  People  Leam,  p.  4.  Differences  between  the  organization  of 
knowledge  in  novices  and  experts  help  to  explain  why  novices  often  base  their  problem  solving 
strategies  on  a  problem’s  surface  features  while  experts  see  through  surface  features  to  more 
important  underlying  principles.  For  an  example  involving  how  novices  and  experts  solve 
inclined  plane  physics  problems  see  Bransford,  et  al..  How  People  Learn,  pp.  38-40. 

60  Ibid.,  p.  4.  and  pp.  45-48. 

61  Bransford  et  al.,  How  People  Learn,  p.  10.  For  a  different  twist  on  this  dealing  with  how 
people  leam  from  and  understand  stories,  see  Roger  C.  Schank,  Tell  Me  a  Story:  A  New  Look  at 
Real  and  Artificial  Memory  (New  York,  NY:  Charles  Scribner’s  Sons,  1990),  pp.  56-83. 
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suggests  that  a  “one-size-fits-all”  approach  to  learning  may  fail  to  accommodate  this  progressive 
self  assessment  and  may  not  properly  engage  each  individual  learner’s  existing  knowledge,  since 
each  learner  will  have  a  slightly  different  background.62 

Bransford  et  al.,  highlight  three  key  findings  that  research  has  revealed  about  learning: 

1.  Students  come  to  the  classroom  with  preconceptions  about  how  the 
world  works.  If  their  initial  understanding  is  not  engaged,  they  may  fail 
to  grasp  the  new  concepts  and  information  that  are  taught,  or  they  may 
learn  them  for  the  purposes  of  a  test  but  revert  to  their  preconceptions 
outside  the  classroom. 

2.  To  develop  competence  in  an  area  of  inquiry,  students  must:  (a)  have 
a  deep  foundation  of  factual  knowledge,  (b)  understand  facts  and  ideas 
in  the  context  of  a  conceptual  framework,  and  (c)  organize  knowledge  in 
ways  that  facilitate  retrieval  and  application. 

3.  A  “metacognitive”  approach  to  instruction  can  help  students  learn  to 
take  control  of  their  own  learning  by  defining  learning  goals  and 
monitoring  their  progress  in  achieving  them.63 

Although  these  key  findings  appear  to  be  directed  toward  children,  the  authors  note  that  they 
apply  to  adult  learning  as  well.  They  even  use  examples  of  how  professional  development 
programs  for  teachers  often  violate  many  of  these  principles!64 

There  are  several  implications  based  on  each  of  these  principles  for  a  program  to  transfer 
expertise-related  knowledge  between  generations  of  intelligence  analysts.  First,  since  the 
novices  who  will  be  learning  the  knowledge  will  generally  have  different  educational 
backgrounds  and  work  experiences,  the  learning  experience  must  be  geared  towards  engaging 
them  at  their  level  and  in  terms  they  can  understand.  Second,  the  knowledge  elicited  from  the 
experts  must  capture  the  factual  knowledge,  information  on  the  contexts  in  which  it  applies,  and 
a  system  for  organizing  and  making  connections  within  the  knowledge.  Third,  information  on 


62 

Bransford  et  al..  How  People  Learn ,  p.  12 
62  Ibid.,  pp.  14-18. 

64  Ibid.,  pp.  26-27. 
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the  typical  stumbling  blocks  and  barriers  to  learning  the  knowledge  must  be  discovered  to  help 
learners  get  through  these  difficulties.  Fourth,  the  knowledge  captured  must  include  information 
learners  can  use  to  gauge  their  own  understanding. 

We  have  only  touched  on  a  few  of  the  most  important  lessons  from  the  science  of 
learning  and  will  rely  on  additional  insights  from  this  discipline  in  the  next  section.  For  now,  it 
is  time  to  consider  some  knowledge  preservation  projects  that  government  organizations  are 
using  to  help  extract  expertise-related  knowledge  from  their  aging  workforces. 

Knowledge  Preservation  Efforts.  Several  U.S.  government  organizations  are  pursuing 
knowledge  preservation  projects.  Some  projects  have  accumulated  a  considerable  amount  of 
experience  while  others  are  still  on  the  drawing  board.  The  discussion  that  follows  is  not  based 
on  a  comprehensive  survey  of  government  knowledge  preservation  projects,  but  rather  a  brief 
discussion  of  a  few  projects  that  have  valuable  implications  for  the  ONI  case. 

Some  of  the  most  mature  government  knowledge  preservation  projects  are  at  the  U.S. 
Department  of  Energy  (DOE).  These  projects  are  part  of  the  DOE’s  efforts  to  preserve  the 
knowledge  base  needed  to  operate  a  downsized  nuclear  weapons  complex.65  Faced  with  a 
moratorium  on  nuclear  testing  and  cessation  of  designing  and  building  new  nuclear  weapons, 
DOE  must  find  ways  to  ensure  the  integrity  of  the  existing  stockpile  of  nuclear  weapons  and 
preserve  the  knowledge  and  expertise  required  to  resume  nuclear  weapons  development  and 
testing,  should  changing  world  events  drive  the  nation  back  in  this  direction. 

Since  the  early  1990s,  DOE’s  major  focus  has  been  on  science-based  stewardship  of  the 
nuclear  weapons  stockpile.  The  vision  is  to  use  scientific  understanding  and  expert  judgment  to 

65  For  a  recent  and  concise  overview  of  the  problem  facing  DOE  and  some  of  the  progress  being 
made  at  Los  Alamos  National  Laboratory  see  Evan  Ratcliff,  “Inside  Nuke  University,”  Wired, 
March  2002,  66-70. 
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predict,  identify,  and  correct  problems  in  the  stockpile.  Non-nuclear  testing  and  computer 
simulation  technologies  are  expected  to  play  a  significant  role  in  this  effort.66  Knowledge 
preservation  is  a  key  part  of  the  overall  Stockpile  Stewardship  Program.  The  goal  is  to  capture 
and  preserve  the  accumulated  knowledge  of  the  nuclear  weapons  complex. 

Elements  of  knowledge  preservation  at  DOE  include  document  archiving  and  knowledge 
capture  activities.  Document  archiving  activities  range  from  cataloging  and  scanning  documents 
for  electronic  access  to  analysis  and  reinterpretation  of  past  nuclear  and  non-nuclear  test  data  to 
help  develop  new  computer  simulation  models.  Knowledge  capture  activities  consist  of 
recording  interviews  with  retiring  or  retired  analysts.  These  activities  fall  under  the  Nuclear 
Weapons  Information  Project.67  Since  knowledge  capture  is  one  of  the  main  thrusts  of  this 
study,  the  following  discussion  focuses  on  the  interviewing  of  experts. 

At  DOE,  interviews  have  been  conducted  with  individuals  and  panels  of  experts  with 
weapons  knowledge.  The  objective  has  been  to  capture  valuable  knowledge  that  has  not  been 
documented  in  any  other  way.  Some  information  that  might  have  otherwise  been  documented 
was  not  committed  to  writing  because  of  security  concerns  over  its  sensitive  nature  and  because 
the  frenetic  pace  of  the  weapons  program  in  the  1950s  and  1960s  did  not  allow  time  for  detailed 
documentation  to  be  created 68 


66  DOE  Office  of  Inspector  General,  The  U.S.  Department  of  Energy's  Efforts  to  Preserve  the 
Knowledge  Base  Needed  to  Operate  a  Downsized  Nuclear  Weapons  Complex  (DOE/IG-0428, 
October,  1998)  <http://www.ig.doe.gov/pdf/ig-0428.pdf>  [May  13, 2002] 

67  Preserving  Nuclear  Weapons  Information ,  Lawrence  Livermore  National  Laboratory  web 
page,  <http://www.llnl.gov/str/Lowns.html>  [April  27, 2002]. 

68  Dan  Stober,  “Nuclear  Secrets:  Steal  This!,”  Bulletin  of  the  Atomic  Scientists  55  (1999):  14-16. 
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Examples  of  the  type  of  information  recorded  in  these  interviews  include  problems 
encountered  in  designing  weapons,  reasons  for  design  choices,  operational  safety,  and  the  art  of 
weapons  design  and  production.69 

It  is  enlightening  to  compare  the  knowledge  capture  activities  at  several  DOE  sites. 
Table  1  summarizes  these  activities.  The  information  dates  from  August  1998  and  does  not 
reflect  the  current  status  of  these  projects. 


Table  1 

Comparison  of  Knowledge  Capture  Activities  at  Several  DOE  Sites 


Site 

Sessions 

Artifacts 

Digital 

Video 

Archive? 

Comments 

Sandia 

132  individual 
&  panel 
discussions 

>  870 
video 
tapes 

Yes 

$4.5  million 
Estimated  costs 

60  more  video 
tapes  planned 

Los  Alamos 

24  panel 
sessions 

video 

tapes 

No 

No  plans  for  digital 
video  archive 
$400K  estimated 
costs 

Livermore 

200  hours 

video 

tapes 

Yes 

Pantex 

not  specified 

paper 

No 

Abandoned 

videotaping 

Y-12 

239 

interviews 

paper  & 

electronic 

transcript 

No 

Effort  completed  in 
four  years 

SOURCE:  DOE,  Efforts  to  Preserve  the  Knowledge  Base,  (1998),  pp.  4-6. 
and  <http://www.bmpcoe.org/bestpractices/internal/oakri/oakri  88.html> 
[May  20,  2002]. 


69  DOE,  Efforts  to  Preserve  the  Knowledge  Base,  pp.  1-2. 


The  method  and  extent  of  recording  interviews  with  experts  varies  across  the  DOE  sites. 
Although  videotaping  interviews  and  panel  discussions  is  a  common  practice  at  most  of  the  sites, 
officials  at  Pantex  abandoned  videotaping  in  favor  of  documenting  interviews  on  paper  because 
they  believed  that  “videotapes  did  not  provide  value-added  information.”70  Pantex’ s  experience 
with  video  highlights  the  importance  of  pre-  and  post-interview  activities  to  ensure  the 
knowledge  captured  is  worth  preserving,  it  is  captured  in  a  form  that  fosters  learning  and 
transfer,  and  it  is  suitably  indexed  for  efficient  search  and  retrieval.  The  Pantex  view  of  video 
may  have  been  more  positive  had  they  approached  their  video  capture  efforts  from  more 
knowledge-  and  learner-centered  perspectives. 

In  all  of  these  efforts,  it  is  not  clear  to  what  extent  these  videos  have  been  used  by 

learners  and  what  approaches,  if  any,  have  been  used  to  structure  the  video  content  toward  the 
needs  of  learners. 

Two  similarities  come  to  mind  when  comparing  DOE’s  knowledge  preservation  problem 
with  that  of  ONI.  First,  both  ONI  and  DOE  share  concerns  over  the  security  and  access  controls 
required  for  sensitive,  compartmented  information.  ONI  can  benefit  and  learn  from  the 
development  and  testing  experience  at  DOE.  Work  is  progressing  on  a  “pyramidal”  need-to- 
know  scheme  where  individuals  at  the  top  have  access  to  most  of  the  information  while 
individuals  lower  on  the  pyramid  are  only  authorized  access  to  information  in  a  particular 
domain  or  about  a  specific  weapon  system.71 

Second,  experts  in  both  organizations  have  had  to  safeguard  classified  information 
resulting  in  a  “disincentive  to  write  or  keep  technical  papers  that  would  then  become  the  author’s 

70  Ibid.,  pp.  4-5. 

71  Stober,  “Nuclear  Secrets.” 
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responsibility  to  keep  secret”  and  both  organizations  maintained  a  breakneck  pace  to  keep  ahead 
of  the  Soviet  Union  during  the  Cold  War.72  For  these  reasons,  experts  at  ONI  may  have  not 
completely  documented  important  knowledge. 

One  important  difference  between  ONI  and  DOE’s  knowledge  preservation  problems  is 
that  DOE  is  concerned  with  trying  to  preserve  design  and  test  knowledge  in  case  it  is  needed  in 
the  future  as  it  shifts  to  science-based  stewardship  of  the  U.S.  nuclear  weapon  stockpile  while  at 
ONI  the  need  is  for  uninterrupted  analytical  capability.  The  emphasis  at  DOE  is  “downloading” 
essential  information  from  expert’s  heads  for  future  scientists.  For  ONI,  downloading  may  be 
desirable,  but  the  most  important  need  is  to  get  knowledge  into  new  analysts’  heads  without 
experiencing  any  gaps  in  capability. 

DOE’s  knowledge  preservation  projects  offer  three  lessons.  First,  a  well  planned 
knowledge  preservation  program  must  take  into  consideration  the  needs  of  the  learners  that  will 
use  the  knowledge  captured  on  video.  These  consideration  should  shape  the  content  of  the  video 
sessions.  Second,  ONI  can  learn  from  and  take  advantage  of  the  security-related  experience  with 
technology  gained  in  DOE’s  knowledge  preservation  projects.  Third,  ONI  must  emphasize  the 
direct  transfer  of  knowledge  between  analysts’  heads  over  “downloading”  because  of  the  need  to 
maintain  uninterrupted  intelligence  analytical  capability. 

Officials  at  the  Naval  Air  Systems  Command  (NAVAIR)  are  also  concerned  about 
knowledge  preservation.  They  have  plans  for  a  pilot  project  and  have  videotaped  interviews 
with  one  individual.  They  captured  three  hours  of  video  in  an  attempt  to  elicit  Test  and 
Evaluation  (T&E)  knowledge  from  a  retiring  manager  at  NAVAIR.  Although  the  intent  was  to 
capture  technical  knowledge,  the  project’s  knowledge  workers  were  disappointed  with  the 

72  Ratliff,  “Inside  Nuke  University.” 
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content  they  captured.  Instead  of  technical  details  about  T&E  projects,  the  video  revolved 
primarily  around  management  concerns  and  issues.  The  expert  they  interviewed  had  worked  in 
T&E  in  the  past,  but  spent  the  last  several  years  as  a  senior  executive.73 

Although  NAVAIR’s  knowledge  preservation  efforts  are  still  in  their  infancy,  the  results 
of  their  experiment  with  videotaping  offer  at  least  two  lessons.  First,  it  is  not  clear  the  right 
person  was  identified  for  interviews  to  capture  T&E  technical  knowledge.  Also,  the  reason  T&E 
was  selected  over  other  air  warfare  domains  is  not  clear.  An  obvious  lesson  from  this  experience 
is  to  find  the  right  experts  and  have  an  end  use  for  the  knowledge  in  mind  before  videotaping 
interviews.  Second,  from  a  discussion  with  a  knowledge  worker  familiar  with  the  videotaping 
sessions,  it  is  clear  that  the  interviewers  were  not  trained  in  conducting  interviews  and  did  not 
use  interview  techniques  designed  to  elicit  expert  knowledge.74  The  lesson  here  is  that 
interviewers  can  benefit  from  gaining  some  experience  and  training  in  interview  techniques. 

Finally,  it  is  not  a  good  idea  to  try  and  draw  too  much  from  or  be  too  critical  of  an  effort 
that  was  clearly  an  experiment.  Such  experimentation  is  a  critical  element  of  successful 
knowledge  management  programs  (remember  Fahey  and  Prusak’s  deadly  sin  number  nine: 
“Failing  to  recognize  the  importance  of  experimentation”). 

Summary.  In  this  section,  some  lessons  from  knowledge  management,  the  science  of 
learning,  and  government  knowledge  preservation  projects  were  discussed.  Knowledge 
management  is  a  valuable  source  of  ideas  because  the  theory  of  organizational  knowledge 
creation  and  resulting  guidance  has  been  synthesized  from  hundreds  of  projects.  From  the  theory 
of  organizational  knowledge  creation,  the  central  role  that  tacit  knowledge  conversion  and 

73 

74  Sherry  L.  Mitchell,  telephone  conversation  with  author,  30  April  2002. 

Such  techniques  include  methodologies  for  cognitive  task  analysis  discussed  in  the  next 
section. 
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human  interaction  play  in  creating  new  products  and  services  can  also  be  seen  to  apply  in  the 
transfer  of  knowledge  between  people.  The  fundamentals  are  the  same — people  embedded  in 
organizations  wrestling  with  tacit  and  explicit  knowledge.  The  knowledge  management  project 
guidelines  and  pitfalls  provide  some  “rules-of-thumb”  to  consider  when  constructing  a  model  of 
knowledge  preservation. 

Results  from  the  science  of  learning  indicate  that  knowledge  capture  activities  are  only 
one  part  of  the  knowledge  transfer  process.  Unless  the  learner’s  needs  are  used  to  guide  the 
knowledge  capture  activities,  important  information  about  the  applicable  contexts,  the  best  ways 
to  organize  the  knowledge  to  solve  problems,  and  the  cues  learners  can  use  to  monitor  the 
progress  of  their  own  learning  may  be  missed.  Other  lessons  that  can  enhance  learning  and 
knowledge  transfer  include  the  need  to  engage  a  learner’s  initial  understanding  and 
preconceptions  and  the  need  to  identify  common  barriers  and  obstacles  to  learning.  Lessons  from 
government  knowledge  preservation  projects  highlight  the  value  of  planning  and  preparation  to 
identify  the  right  sources  of  knowledge  and  establish  priorities  for  the  knowledge  to  be  captured. 
Lessons  from  DOE’s  experience  recording  hundreds  of  interviews  and  using  a  networked  digital 
video  archive  with  security  access  controls  can  be  used  by  ONI  to  overcome  similar  challenges. 
The  video  experiment  at  NAVAIR  highlights  the  importance  of  identifying  the  right  experts, 
identifying  an  end  use  for  the  knowledge  captured,  and  using  interview  techniques  to  guide  the 
discussion  and  probe  critical  areas  to  help  elicit  knowledge. 

This  brief  background  survey  of  industry,  science,  and  government  suggests  the  potential 
benefits  from  a  framework  or  model  around  which  a  knowledge  preservation  program  can  be 
constructed.  In  the  next  section  a  knowledge  preservation  model  is  developed  and  presented. 
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SECTION  V 


KNOWLEDGE  PRESERVATION  MODEL 

Making  things  look  easy  is  hard.  Clarity  requires  depth. 

—  Tor  Norretranders75 

Introduction.  This  section  explains  a  proposed  knowledge  preservation  model  for 
organizations  faced  with  experts  who  have  retired  or  will  retire  in  the  near  future.  This  model 
draws  on  ideas  from  knowledge  management  and  the  science  of  learning  that  were  presented  in 
the  previous  section.  Unlike  conventional  knowledge  preservation  approaches  which  focus 
almost  exclusively  on  knowledge  capture  activities  and  on  the  archiving  of  data  and  information, 
the  model  presented  here  focuses  on  the  end-to-end  transfer  of  knowledge  from  the  heads  of 
retiring  experts  to  the  heads  of  their  replacements. 

This  section  begins  by  constructing  a  simplified  model  of  end-to-end  knowledge  transfer 
to  serve  as  the  foundation  for  an  overall  knowledge  preservation  model.  This  simplified  model 
and  the  rationale  behind  it  will  then  be  used  to  infer  some  implications  for  knowledge 
preservation  and  build  a  case  for  adopting  a  story-centered  approach  to  learning.  Next  a  three¬ 
pronged  model  for  knowledge  preservation  is  described.  Finally,  highlights  are  summarized  and 
the  stage  is  set  for  considering  how  to  put  the  model  into  action  at  ONI  in  the  next  section. 


Tor  Norretranders,  The  User  Illusion:  Cutting  Consciousness  Down  to  Size,  trans.  Jonathan 
Sydenham  (New  York,  NY:  Penguin  Putnam,  Inc.,  1998),  p.  79. 
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A  Simplified  Model  of  Knowledge  Transfer.  One  way  to  view  knowledge  transfer  is  to 
think  in  terms  of  a  cybernetic  communication  model.76  This  model  consists  of  a  sender  that  acts 
as  a  source  to  encode  and  send  messages  over  a  communications  channel  to  a  receiver  which 
decodes  and  receives  the  messages.  Noise,  a  “general  term  for  anything  that  tends  to  produce 
errors  in  transmission,”  can  also  degrade  performance  as  it  is  added  to  the  messages.77  These 
model  elements  and  their  relationships  are  depicted  in  figure  4.  This  model  was  used  to 


Figure  4 

Cybernetic  Communication  Model 


Sender  Messages  Receiver 


SOURCE:  Adapted  from  Ash  (1965),  p.  1. 

formulate  information  theory  and  has  led  to  the  development  of  today’s  communication  systems 
and  other  products  including  compact  disk  players.  Although  a  powerful  model,  its  use  here  is  as 


76  Claude  Shannon  and  Norbert  Weiner  developed  much  of  the  theory  that  led  to  the  popularity 
of  this  model.  The  ideas  behind  its  use  here  come  from  Robert  B.  Ash,  Information  Theory 
(New  York,  NY:  Dover  Publications,  Inc.,  1965)  and  W.  Ross  Ashby,  An  Introduction  to 
Cybernetics  (London,  UK:  Chapman  &  Hall,  Ltd.,  1957) 

77  Ash,  Information  Theory,  p.  1. 
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a  point  of  departure  to  develop  a  more  comprehensive  model  of  knowledge  transfer  between 
people.78 

The  cybernetic  communication  model  is  very  general  in  the  sense  that  the  rules  governing 
properties  like  the  channel  capacity  hold  for  many  different  circumstances.79  For  example,  the 
sender  could  be  a  satellite  ground  station  sending  messages  over  a  channel  consisting  of  the  path 
from  the  ground  station  to  a  geosynchronous  satellite  back  down  to  a  receiver  on  a  ship  at  sea. 
Or  the  sender  could  be  musician  in  a  recording  studio  whose  song  is  recorded  and  then 
transmitted  through  the  channel  consisting  of  a  compact  disk  which  is  in  turn  played  by  a  music 
fan  who  serves  as  the  receiver. 

So  it  would  seem  logical  to  try  to  extend  this  model  to  the  case  of  knowledge  transfer 
where  the  sender  is  a  retiring  expert  transmitting  knowledge  in  the  form  of  messages  to  a 
someone  designated  to  learn  the  retiring  expert’s  craft.  Here  the  channel  could  consist  of  direct 
verbal  conversations,  video  tape,  documents,  procedures,  test  results,  and  other  suitable  media 
and  artifacts  for  transmitting  knowledge.  For  this  model  to  work  the  receiver  must  understand 
how  to  decode  the  messages  and  it  must  understand  what  the  symbols  mean.  For  a  satellite 


A  more  comprehensive  model  has  a  better  “relation-structure”  to  the  process  it  models  and 
ence  does  a  better  job  of  explanation.  Johnson-Laird  uses  an  example  of  how  a  person’s  mental 

5®  Umitrd  t0  "  b°X  Aat  di$playS  m°ving  pictures  ^ith  sound- 

much  nVhp/^f  f  ^ ^  ?  ad®quate  for  many  Pe°Ple>  but  a  television  repairman  would  have  a 
much  ncher  mental  model  of  a  television  set  that  includes  an  electron  beam  being 

electromagnetically  steered  m  a  raster  pattern  to  form  the  images  on  the  screen.  This  level  of 

Pi  ced "" i,t0 r^11  ^ thC ^ " dist°rted  When - exte™almagnet is 
p  ced  near  the  set.  One  could  also  have  an  even  more  detailed  model  that  included  perhaps  the 

quantum  mechanical  effects,  but  this  extra  detail  would  not  add  significantly  to  the  explanatory 

tocfon  of  the  repairman’s  model.  See  P.  N.  Johnson-Laird.  MeZ  Model: 2! ^ 

Cognitive  Science  of  Language,  Inference,  and  Consciousness  (Cambridge,  MA*  Harvard 

Ashby,  An  Introduction  to  Cybernetics,  p.  155. 
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communication  system,  the  engineers  who  design  the  system  understand  this  and  build  their 
receivers  accordingly.  For  compact  disks,  music  lovers  just  listen  to  music  over  their 
headphones  and  enjoy  the  emotional  states  aroused.  However,  messages  encoded  with  expertise- 
related  knowledge  are  not  ordinary  messages,  and  a  learner  does  not  automatically  know  how  to 
understand  and  decode  knowledge  from  a  retiring  expert.  Some  adjustments  are  needed  to  make 
a  more  useful  model. 

Part  of  the  difficulty  in  applying  the  cybernetic  communication  model  to  knowledge 
transfer  stems  from  the  fact  that  the  “information  content”  in  information  theory  is  independent 
of  meaning.  The  information  content  of  a  message  is  a  measure  of  the  uncertainty  in  the 
message  based  on  a  calculation  involving  the  number  of  possible  states  it  could  assume.  By  this 
measure,  one  message  heavily  ladened  with  meaning  could  have  the  same  information  content  as 
another  message  that  is  pure  nonsense.80 

How  does  one  then  account  for  meaning  in  a  message?  One  way  is  through  the  ease  of 

its  predictability.  Consider  a  disordered  message,  one  composed  entirely  of  random  noise.  The 

exact  value  of  such  a  message  at  any  instant  is  uncertain.  However,  such  a  message  is  still  fairly 

predictable.  In  many  cases,  it  can  be  completely  described  by  a  few  statistical  parameters  such 

as  the  values  of  its  mean  and  standard  deviation.  Now  consider  an  ordered  message,  one 

composed  of  a  fixed,  repeating  pattern.  This  message  can  be  easily  predicted  once  one  knows 

the  pattern.  This  implies  that  the  notion  of  meaning  or  value  in  a  message  must  somehow  be  tied 

up  in  something  that  lies  between  order  and  disorder. 

The  value  of  a  message  thus  appears  to  reside  not  in  its  information  (its 
absolutely  unpredictable  parts),  nor  in  its  obvious  redundancy  (verbatim 
repetitions,  unequal  digit  frequencies),  but  rather  in  what  might  be  called 

80  Norretranders,  The  User  Illusion,  p.  99. 
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its  buried  redundancy — parts  predictable  only  with  great  difficulty.  In 
other  words,  the  value  of  a  message  is  the  amount  of  mathematical  or 
other  work  plausibly  done  by  its  originator,  which  the  receiver  is  saved 
from  having  to  repeat81 

Charles  Bennett,  a  physicist  and  pioneer  in  the  study  of  complexity,  called  this  plausible 
amount  of  work  in  a  message  its  “logical  depth”  and  associated  it  with  a  measure  of  complexity. 

“Complexity,  m  this  sense,  is  a  measure  of  how  hard  it  is  to  put  something  together  starting  from 
elementary  pieces.”82 

The  key  idea  is  to  make  the  connection  that  the  meaning  or  value  of  a  message  lies  in 

something  that  is  “missing”  from  the  physical  message.  Norretranders  makes  the  following 

connection  between  logical  depth  and  missing  information: 

Logical  depth  is  a  measure  of  the  process  that  leads  to  a  certain  amount 
of  information,  rather  than  the  amount  of  information  that  is  produced 
and  is  transmitted.  Complexity  or  meaning  is  a  measure  of  the 
production  process  rather  than  the  product,  the  work  time  rather  than  the 
work  result.  The  information  discarded  rather  than  the  information 
remaining 

It  is  not  the  face  value  of  the  information  that  defines  its  depth,  but  how  difficult  the 
information  was  to  generate.  For  example,  “$1.59  per  pound”  does  not  contain  much 
information  by  itself.  However,  if  I  am  shopping  in  the  supermarket  and  I  need  to  find  the  best 
buy  between  two  different  brands  in  three  package  sizes,  I  need  to  calculate  the  per  unit  price  for 
each  option.  These  calculations  take  time  and  effort.  Information  is  discarded  along  the  way. 
The  price  of  a  package,  the  number  of  ounces,  the  conversion  factor  from  ounces  to  pounds  are 
all  no  longer  needed  to  make  the  comparison  once  you  have  the  answer.  When  the  grocer 


8 1 

Charles  Bennett,  “Logical  Depth  and  Physical  Complexity,”  quoted  in  Norretranders,  The 

User  Illusion,  pp.  77-78. 

82  *  1 1 

Heinz  R.  Pagels,  The  Dreams  of  Reason:  The  Computer  and  the  Rise  of  the  Sciences  of 
Complexity  (New  York,  NY:  Simon  &  Schuster,  Inc.,  1 988),  p.  67. 

Norretranders,  The  User  Illusion,  p.  79. 
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provides  the  per  unit  prices  on  the  shelf  next  to  each  item,  he  has  saved  me  a  certain  amount  of 

04 

computational  time. 

The  concept  of  logical  depth  is  not  restricted  to  supermarket  aisles.  One  can  think  of 
logical  depth  in  terms  of  human  conversations.  Norretranders  illustrates  this  point  with  an 
example  of  the  communication  between  the  French  novelist  Victor  Hugo  and  his  publisher. 
While  vacationing  after  writing  the  novel  Les  Miserables,  Hugo  became  curious  about  how  well 
his  new  book  was  selling.  He  cabled  his  publisher  with  a  message  consisting  of  a  single  question 
mark:  “?”  His  publisher  replied  with  a  single  exclamation  point:  “!”  Hugo’s  book  was  doing 
very  well  indeed.85  This  exchange  would  be  meaningless  without  the  shared  context  between 
Hugo  and  his  publisher.  With  their  shared  context,  they  are  able  to  discard  an  extreme  amount  of 
information. 

. . .  our  words  contain  more  than  can  be  measured  in  bits.  Because  it  is 
not  what  we  say  to  each  other  every  day  that  establishes  all  the  meaning 
and  beauty  and  truth  our  everyday  conversations  contain;  it  is  everything 
we  think  before  we  speak  . . .  there  must  be  something  else  that  really 
counts:  the  real  source  of  beauty,  truth,  and  wisdom.  The  ironic  thing  is 
that  this  “else”  can  be  described  as  the  information  we  have  got  rid  of: 
exformation.  Meaning  is  information  that  has  been  discarded: 
information  that  is  no  longer  present  and  no  longer  needs  to  be.86 

Norretranders  compares  the  relationship  between  information  and  meaning  to  the 
relationship  between  money  and  wealth.  The  real  value  of  money  lies  in  getting  rid  of  it.  That 
is  what  provides  the  substance  of  real  wealth.  It  is  not  having  a  lot  of  money  per  se,  it  is  having 
the  possessions  and  trappings  that  money  can  buy.87 


u  Ibid.,  p.  79. 
85  Ibid.,  p.  91. 

Ibid.,  p.  98. 
87  Ibid.,  p.  98. 


The  information  in  a  message  that  gets  explicitly  discarded  is  called  exformation.  In  one 
sense  the  value  and  the  worth  of  a  message-its  meaning-lies  not  in  its  infoimation  but  its 
exfoimation.  In  other  words,  the  information  that  was  discarded  along  the  way. 

If  exformation  is  explicitly  discarded  information,  then  exactly  what  does  a  word  mean? 
For  example,  what  do  we  know  when  we  know  what  a  dog  is?  This  is  something  that  most 
people  take  for  granted.  However,  when  one  starts  to  really  think  about  what  seems  a  simple 
concept,  for  instance,  defining  a  word  like  “dog,”  it  becomes  clear  that  how  the  mind  works  is 
actually  quite  complicated.  For  example,  one  could  define  a  dog  as  an  animal  having  four  legs, 
fur,  a  tail,  and  the  ability  to  bark.  The  problem  is  that  for  every  defining  characteristic  one  can 
name,  there  is  an  exception.  Consider  a  dog  that  has  lost  a  leg  or  lacks  a  tail.  It  is  still  a  dog  and 
one  can  easily  recognize  such  an  animal  as  a  dog.  Ludwig  Wittgenstein,  a  twentieth  cenhuy 
philosopher,  introduced  the  concept  of  family  resemblance  to  explain  how  people  get  around 
these  difficulties  in  everyday  practice.  lust  as  one  can  recognize  the  members  of  a  human  family 
by  similarities  in  properties  such  as  eye  color,  height,  expression,  and  temperament,  so  can  one 
recognize  a  dog  by  its  similar  properties  of  “dogness.”88 

The  point  of  this  digression  into  the  meaning  of  a  word  is  that  there  is  a  lot  of  processing 
gomg  on  behind  the  scenes,  of  which  people  are  unaware,  on  both  the  transmit  and  receive  side 
of  a  conversion  when  they  use  a  common  word  like  “dog.”  Communicating  with  these  common 
words  is  possible  because  most  people  have  developed  an  understanding  of  what  these  words 
mean.  What  about  less  common  words,  concepts,  and  ideas  that  a  person  on  the  receiving  end  of 
a  conversation  is  less  likely  to  be  familiar  with?  This  is  where  the  notion  of  logical  depth  and 


88 

For  an  in-depth  discussion  of  family  resemblance 
191  and  Reisberg,  Cognition,  p.  273. 


see  Johnson-Laird,  Mental  Models,  pp.  187- 
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exformation  can  be  used  to  modify  the  cybernetic  communication  model  by  incorporating  some 
relation-structure  to  account  for  perception  and  mental  processing. 

Consider  that  your  five  senses  and  the  mental  processing  associated  with  them  consume 
about  eleven  million  bits  per  second  in  information  terms,  while  language  and  the  consciousness 
that  process  it  occupy  only  about  forty  bits  per  second.  These  numbers,  although  approximate, 
are  not  speculation;  they  have  been  substantiated  through  many  scientific  experiments.  A  vast 
amount  of  information  is  discarded  when  we  go  from  experiencing  life  in  person  to  explaining  it 
in  words.89 

Humans  also  must  discard  a  lot  of  information  when  they  tell  a  story.  Mental  imagery, 
sensations,  and  intuition  all  peel  off  in  the  process  of  thinking  that  ends  in  the  spoken  or  written 
word.  “We  can  know  more  than  we  can  tell,”  said  Polanyi.90 


RQ 

Norretranders,  The  User  Illusion,  p.  126. 
90  Polanyi,  The  Tacit  Dimension ,  p.  4. 


51 


These  ideas  can  be  used  to  adjust  the  cybernetic  communication  model  of  figure  4.  Both 

the  sender  and  receiver  are  replaced  by  what  Tor  Norretranders  calls  “the  tree  of  talking.”91  This 

is  shown  conceptually  in  figure  5.  Once  again,  imagine  the  sender  to  be  the  retiring  expert  and 

the  receiver  to  be  the  aspiring  learner.  Here  is  how  Norretranders  describes  the  process: 

The  movement  proceeds  from  the  left-hand  treetop  down  to  its  root 
through  the  forest  floor  and  up  the  right-hand  tree.  On  the  left,  a  lot  of 
information  is  compressed  via  the  discarding  of  information  the 
production  of  exformation.  Thoughts  are  composed  into  words.  We  can 
ca  this  incitation.  On  the  right,  the  limited  information  in  the  words  is 
received.  This  is  unfurled  into  more  information.  We  can  call  this 
second  process  excitation. 


Figure  5 

Tree  of  Talking  Communication  Model 

Bits  per 


Norretranders,  The  User  Illusion,  pp.  91-123 
92  lbid.,pp.  110-11. 
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N0rretranders  points  out  that  the  model  applies  beyond  language  to  music,  for  example, 
which  has  its  own  tree.  He  also  cites  cognitive  neuroscience  and  brain  studies  which  provide 
evidence  that  more  than  guesswork  lies  behind  the  notion  of  these  trees. 

Now  we  have  a  conceptual  model  of  knowledge  transfer  from  which  to  build  a  model  of 
knowledge  preservation  that  accommodates  some  of  the  lessons  from  the  previous  section, 
including  the  need  to  engage  learners’  previous  knowledge  and  the  need  to  provide  context  for 
information. 

Implications  for  Knowledge  Preservation.  The  first  inference  to  draw  from  the  tree  of 
talking  communication  model  is  the  need  for  an  expert  to  throw  away  information  in  order  to 
communicate  knowledge  to  a  learner.  The  expert  needs  to  produce  exformation.  The 
information  that  is  discarded  needs  to  be  carefully  selected  so  that  the  remaining  message 
conveys  the  essence  of  the  ideas  behind  the  knowledge  and,  at  the  same  time,  refers  to 
information  which  can  be  easily  excited  in  the  mind  of  a  learner.  This  occurs  naturally,  for 
example,  when  someone  composes  a  story  to  tell  another  person.  As  someone  relates  the  gist  of 
a  story,  he  necessarily  takes  an  intentional  stance  to  consciously  elaborate  on  some  details  while 
glossing  over  others.94  This  idea  relates  to  some  lessons  from  the  science  of  learning.  Since  it  is 
the  context  of  the  factual  knowledge,  the  fluent  retrieval  of  pertinent  facts,  and  how  an  expert’s 
knowledge  is  organized  that  most  influence  the  difference  in  competency  between  novices  and 


93  Ibid.,  pp.  111-20. 

94  For  a  description  of  how  the  gist  of  a  story  is  transformed  into  a  realization  in  language  see 
Schank,  Tell  Me  a  Story,  p.  25.  For  more  on  the  intentional  stance  and  its  relationship  to 
meaning  see  Daniel  C.  Dennett,  The  Intentional  Stance  (Cambridge,  MA:  The  MIT  Press,  1987), 
p.l7,p.  132. 
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experts,  it  is  information  relevant  to  these  details  that  must  be  conveyed  while  other  information 
is  explicitly  discarded.95 

The  second  inference  to  draw  from  the  tree  of  talking  model  is  the  need  to  engage  a 
learner  s  existing  knowledge.  Learning  is  not  effective  unless  the  expert’s  message  resonates 
with  and  excites  relevant  trees  in  learner’s  mind. 

A  good  communicator  does  not  think  only  of  himself;  he  also  thinks 
about  what  the  receiver  has  in  his  head.  It  is  not  enough  for  the 
explicitness  of  the  information  to  refer  to  some  information  in  the 
sender  s  head  if  that  information  does  not  somehow  lead  to  the  correct 
associations  by  the  receiver.96 

This  idea  relates  back  to  the  lessons  from  the  science  of  learning  that  new  knowledge  must  be 
constructed  from  and  combined  with  existing  knowledge.  This  helps  to  explain  why  redundancy 
is  one  of  Nonaka  and  Takeuchi’s  enabling  conditions  for  organizational  knowledge  creation. 

Redundancy  provides  a  common  vocabulary  and  mental  models  for  people  to  use  in  relating  to 
one  another. 

The  third  inference  is  the  need  for  rich  “high  bandwidth”  interaction  between  people. 

One-on-one  interaction  between  an  expert  and  a  learner  is  the  most  direct  form  of  this.  Video 

teleconferencing  is  another  form  high  bandwidth  interaction,  although  it  is  less  rich  than  one-on- 

one  interaction.  Recorded  video  is  another  high  bandwidth  form  of  communication,  although  it  is 

hardly  interactive.  An  advantage  of  one-on-one  interaction  is  that  it  provides  an  opportunity  for 

feedback  from  the  learner  that  the  expert  can  use  to  make  adjustments  in  his  message’s 

exformation  and  thereby  better  convey  knowledge.  Also,  one-on-one  interaction  need  not  be 

confined  to  the  low  bandwidth  of  speech  and  writing  but  can  also  take  place  by  watching  the 

expert  at  work  and  socially  by  “getting  to  know  one  another.”  These  higher  bandwidth  forms  of 

95  Refer  to  the  subsection  on  the  science  of  learning  in  Section  IV. 

Norretranders,  The  User  Illusion,  p.  93. 
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interaction  occur  further  up  the  tree  and  can  add  substantially  to  the  quality  of  learning  that  takes 
place. 

The  need  for  high  bandwidth  relates  back  to  Nonaka  and  Takeuchi’s  theory  of  knowledge 
creation.  Their  theory  revolves  around  the  mobilization  and  conversion  of  tacit  knowledge.  In 
both  the  conversion  of  tacit  to  explicit  knowledge  through  extemalization  and  the  conversion  of 
tacit  knowledge  between  individuals  through  socialization,  information  is  discarded  and 

Q7 

exformation  is  produced. 

In  terms  of  the  tree  of  talking  model,  interaction  higher  up  the  tree  can  result  in  shared 
experience  and  a  greater  chance  of  achieving  shared  meaning.  This  idea  relates  back  to  the 
lessons  from  knowledge  management  in  that  socialization  (the  tacit-to-tacit  knowledge 
conversion  process)  makes  use  of  high  bandwidth  connections  as  learners  “share  experiences”  or 
engage  their  five  senses  as  they  “learn  by  doing.”  In  internalization,  the  explicit-to-tacit 
knowledge  conversion  process  also  makes  use  of  high  bandwidth.  This  occurs  in  the  mind  of  the 
learner  as  he  “assimilates”  the  explicit  knowledge  into  mental  “terms  of  his  own”  or  tacit 
knowledge.  Lessons  from  the  science  of  learning  also  relate  to  the  advantages  of  a  high 
bandwidth  learning  channel.  First,  multiple  channels  and  high  bandwidth  connections  provide 
more  opportunities  for  engaging  existing  knowledge  in  the  learner’s  mind.  Second,  they  can  also 
help  with  one’s  ability  to  self- assess  understanding  or  metacognition  by  providing  a  learner  with 
a  “feel”  for  the  subject.  Another  high  bandwidth  way  to  achieve  this  is  through  the  use  of 
computer  simulations  and  experimentation.  For  example,  although  Microsoft  Flight  Simulator™ 
is  not  as  realistic  a  flying  experience  as  actually  going  up  in  an  airplane  behind  the  controls,  it  is 
a  much  richer  experience  flying  the  simulator  than  just  reading  about  it. 

97  See  figures  2  and  3  in  Section  IV  for  a  visual  representation  of  these  processes. 
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Having  constructed  a  simplified  model  of  knowledge  transfer  and  used  it  along  with 
some  of  the  lessons  from  the  previous  section  to  draw  inferences  for  a  knowledge  preservation 
model,  it  is  time  examine  exactly  what  form  an  expert’s  messages  should  take.  It  is  time  to  make 
a  case  for  story-centered  learning. 

A  Case  for  Story-Centered  Learning.9*  The  case  for  using  story-centered  learning  in 
knowledge  transfer  starts  with  establishing  a  connection  between  stories  and  knowledge.  Next, 
the  discussion  turns  to  Schank’s  three  models  for  how  intelligence  and  stories  are  related.  This  is 
followed  by  looking  at  the  role  reminding  plays  in  conversation  and  stories.  This  leads  to  a 
discussion  of  storytelling  as  understanding  and  how  stories  can  be  used  to  get  one’s  point  across. 
The  role  indexing  plays  in  our  ability  to  recall  stories  and  why  they  are  so  easy  to  remember  is 
discussed  as  a  major  advantage  for  learners  and  information  systems  alike.  Next  the  link 
between  sensemakmg  and  stories  is  discussed.  Finally,  stories  are  depicted  as  a  way  to  capture  a 
slice  of  an  expert  s  knowledge  and  experience  in  a  digestible  easy  to  remember  form. 

To  understand  the  connection  between  knowledge  and  stories  consider  that  a  person’s 
knowledge  of  the  world  is  bound  up  in  the  set  of  experiences  he  or  she  has  had.  Not  all  of  a 
person  s  memories  are  stories,  but  the  interesting  ones  and  the  ones  a  person  has  learned 
something  from  are  stories.  For  example,  eating  a  bowl  of  cereal  for  breakfast  on  any  given  day 
is  pretty  much  like  any  other  bowl  of  cereal.  It  is  not  memorable.  However,  if  there  is  a  car 
crash  nearby  one  day  while  eating  breakfast,  the  bowl  of  cereal  becomes  memorable  because  it  is 

98  There  is  an  ample  body  of  literature  on  the  use  of  stories  to  convey  knowledge.  Many  of  the 
ideas  presented  here  come  from  Schank,  Tell  Me  a  Story.  For  the  use  of  stories  to  transfer  tacit 
knowledge  in  organizational  settings  see  Walter  Swap  et  al.,  “Transferring  Tacit  Knowledge 
Assets:  Mentoring  and  Storytelling  in  the  Workplace,”  Journal  of  Management  Information 
Systems,  Summer  2001.  For  the  power  of  stories  for  eliciting  knowledge  from  experts  see  Gary 
Klein,  Sources  of  Power:  How  People  Make  Decisions  (Cambridge,  MA:  The  MIT  Press  1998) 
pp.  177-96. 
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now  part  of  a  story.  Likewise  the  events  a  person  can  recall  to  tell  others  is  limited  to  the  stories 
at  a  person’s  disposal  to  tell.  “In  other  words,  all  we  have  are  experiences,  but  all  we  can 
effectively  tell  others  are  stories.”99 

Schank  uses  three  models  to  explain  how  stories  and  intelligence  are  related.  The  first 
model  is  the  librarian.  If  a  library  patron  asks  for  help  finding  information  on  a  topic,  the 
librarian  may  refer  him  to  a  particular  section  of  the  library  or  the  librarian  may  suggest  a 
particular  book  that  seems  appropriate  but  the  librarian  has  not  actually  read,  or  the  librarian  may 
be  reminded  of  having  read  exactly  the  book  the  patron  is  seeking.  Any  of  these  three  possible 
responses  from  a  human  librarian  to  a  request  for  help  would  be  seem  intelligent.  In  this  model, 
a  librarian’s  intelligence  is  “in  translating  what  we  say  into  the  system  of  labels  by  which  all  the 
books  he  knows  about  have  been  encoded.”100  Here  the  appearance  of  intelligence  is  bound  up 
in  being  able  to  find  the  right  thing. 

The  second  model  is  the  grandfather.  Many  grandfathers  tell  the  same  stories  over  and 
over.  Occasionally,  a  new  and  interesting  story  is  told  which  no  one  remembers  hearing  and  that 
the  grandfather  tells  with  enthusiasm.  A  question  Schank  poses  is:  is  such  a  grandfather  wise? 
He  maintains  the  answer  has  to  do  with  knowing  the  difference  between  intelligence  and 
wisdom. 


We  are  willing  to  ascribe  wisdom  to  those  who  are  not  so  quick  to  draw 
conclusions  but  rather  who  see  many  sides  to  a  problem  and  attempt  to 
draw  reasonable  analogies.  Or  to  put  this  another  way,  wisdom  is  often 
ascribed  to  those  who  can  tell  just  the  right  story  at  the  right  moment  and 
who  often  have  a  large  number  of  stories  to  tell.  Furthermore  these 
stories  rarely  draw  all  the  conclusions  for  the  hearer.  Rather,  they 
present  information,  often  leaving  out  the  final  conclusions.101 


99  Schank,  Tell  Me 
m  Ibid.,  p.  14. 

101  Ibid.,  p.  14. 
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Here  the  appearance  of  intelligence  depends  on  what  is  said  rather  than  what  is  actually 
understood.  Schank  points  out  that  a  computer  with  a  vast  repertoire  of  stories  that  carefully 
selects  the  right  ones  at  the  right  time  can  appear  wise  although  the  computer  may  have  no  actual 
understanding  of  the  stories  it  tells.102 

The  third  model  is  the  logician.  The  idea  here  is  that  every  problem  can  be  broken  down 

into  a  form  that  can  be  decided  by  some  form  of  logic.  However,  people  are  not  set  up  to  think 

like  this,  people  like  to  tell  and  hear  stories  because  stories  provide  context. 

When  a  decision-making  heuristic,  or  rule  of  thumb,  is  presented  to  us 
without  a  context,  we  cannot  decide  the  validity  of  the  rule  we  have 
heard,  nor  do  we  know  where  to  store  this  rule  in  our  memories  Thus 
what  we  are  presented  is  both  difficult  to  evaluate  and  difficult  to 
remember,  making  it  virtually  useless.  People  who  fail  to  couch  what 
they  have  to  say  in  memorable  stories  will  have  their  rules  fall  on  deaf 
ears  despite  their  best  intentions  and  despite  the  best  intentions  of  their 
listeners.  A  good  teacher  is  not  one  who  explains  things  correctly  but 

one  who  couches  explanations  in  a  memorable  (i.e.,  an  interesting) 
format.  b 

What  emerges  from  this  discussion  of  knowledge,  stories,  and  intelligence  can  be 

summed  up  m  the  following  quote:  “Knowledge,  then,  is  experience  and  stories,  and  intelligence 

is  the  apt  use  of  experience  and  the  creation  and  telling  of  stories.”104  Perhaps  what  ONI  needs 

most,  are  grandfathers,  but  with  the  skills  of  both  librarians  and  logicians. 

Stones  can  also  be  very  effective  at  getting  a  point  across.  This  helps  to  explain  why 

techniques  such  as  the  use  of  parables  in  the  Bible  and  ancient  myths  have  had  such  a  long 

history.  Part  of  this  effectiveness  stems  from  the  idea  that  just  being  told  a  rule  does  not  provide 


m  Ibid.,  p.  15. 

103  Ibid.,p.  15. 

104  Ibid.,  p.  16. 
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a  context  nor  produce  the  visual  imagery  and  other  perceptual  associations  that  go  along  with  the 
remindings  that  are  part  of  understanding  a  story.  “Stories  illustrate  points  better  than  simply 
stating  the  points  themselves  because,  if  the  story  is  good  enough,  you  usually  don’t  have  to  state 
your  point  at  all;  the  hearer  thinks  about  what  you  have  said  and  figures  out  the  point 
independently.  The  more  work  the  hearer  does,  the  more  he  or  she  will  get  out  of  your  story.”105 

Another  feature  of  stories  and  one  reason  they  are  easy  to  recall  is  the  many  ways  they 
can  be  indexed  in  memory  or  a  knowledge  retrieval  system.  A  judicious  set  of  well  indexed 
stories  is  one  way  to  capture  some  of  an  expert’s  highest  value  knowledge  in  a  form  that  fosters 
learning  and  retrieval. 

People  find  the  stories  they  wish  to  tell  though  indexing.  Indices  serve  as  labels  that  aid 
in  the  storage  and  retrieval  of  stories.  These  labels  provide  connections.  Reisberg  points  out  that 
connections  are  how  one  memory  triggers  another,  and  this  is  how  people  understand  the  pieces 
of  a  story,  by  connecting  them  with  pieces  of  stories  they  already  are  familiar  with.106  Indices 
can  be  described  in  terms  of  conceptual  actions  that  relate  to  events  in  a  story.  Examples  include 
scripts,  plans,  goals,  and  themes™1  These  indices  are  particularly  well  suited  to  label  expertise- 
related  knowledge.  Schank  describes  how  an  expert  in  military  history  was  found  to  access  his 
knowledge  using  this  type  of  index-like  scheme.  Schank  proposes  that  an  expert  “is  someone 
who  has  a  great  many  stories  to  tell  in  one  particular  area  of  knowledge  and  who  has  those 
stories  indexed  well  enough  to  find  the  right  one  at  the  right  time.”108 


™5  Ibid.,  pp.  11-12. 

106  Reisberg,  Cognition,  p.  153. 

107  Schank,  Tell  Me  a  Story,  pp.  84-114. 
m  Ibid.,  p.  109. 
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This  implies  that  an  indexing  scheme  such  as  that  shown  below  could  be  used  to  elicit 

knowledge  in  the  form  of  stories  from  a  retiring  expert  and  later  be  used  as  an  index  to  retrieve  a 

related  video  clip  of  the  expert  telling  the  story.  An  example  index  might  be: 

Theme:  Approaches  to  S&T  intelligence  analysis 

Goal:  Understand  enemy  weapon  system 

Plan:  Invent  a  dynamic  scenario  that  pits  one  weapon  system  against  another  and 

analyze  each  weapon  system’s  performance 
Result:  Analyst  gets  a  richer  understanding  than  static  numbers  can  provide 

Lesson:  One  learns  more  from  a  dynamic  situations  than  from  static  situations 

The  knowledge  in  the  preceding  example  can  be  indexed  and  captured  in  a  relevant  story 
as  an  example.  Because  stories  are  related  to  and  useful  for  understanding,  they  are  also  useful 
for  sensemaking  (making  sense  out  of  one’s  situation  and  environment).  Stories  can  be  used  to 
organize  information  to  check  for  consistency  and  plausibility  when  trying  to  sort  things  out. 
Can  one  concoct  a  story  to  account  for  the  facts?  If  so,  such  a  story  can  possibly  have  great 
explanatory  power.109 


109  Michael  S.  McBeth,  “Approaches  to  Enhance  Sensemaking  in  Intelligence  Analysis.” 
Unpublished  Study  Report,  Naval  War  College,  Newport,  RI,  14  February  2002. 
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Stories  can  capture  a  slice  of  an  expert’s  knowledge  in  a  context  that  is  easier  for  learners 
to  digest  and  recall  than  disassociated  facts  and  rules.  Figure  6  is  a  conceptual  drawing  that 
shows  an  expert’s  knowledge  represented  as  three  encompassing  ellipses.  Core  knowledge 
represents  all  of  the  underlying  basics.  Domain  knowledge  represents  domain  specific 


Figure  6 

Stories  Capture  a  Slice  of  an  Expert’s  Knowledge 


operated.  Personal  knowledge  represents  knowledge  specific  to  an  individual  that  usually  relates 
to  past  experiences  or  episodes.  An  example  is  a  specific  bit  of  knowledge  captured  in  a  project 
folder  an  analyst  used  to  solve  a  problem  back  in  1995. 

If  the  “Xs”  in  the  figure  represent  particular  nuggets  of  information  that  may  be  relevant 
in  the  future,  stories  can  be  captured  that  slice  across  the  expert’s  knowledge  to  encapsulate 
certain  nuggets  in  a  context.  These  nuggets  can  then  serve  as  one  set  of  indices  for  retrieval  from 
a  learner’s  memory  or  from  a  searchable  knowledge  archive. 
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Since  stories  represent  slices  of  knowledge,  this  picture  indicates  that  one  would  not  want 
to  depend  on  stories  alone  to  build  a  learner’s  core  knowledge  (especially  a  deep  knowledge  of 
the  factual  contents  of  a  subject).  This  basic  level  of  learning  and  understanding  is  usually 
mastered  through  formal  education,  mentoring,  organized  training  programs,  and  self-study.110 
Fortunately,  m  the  case  of  transferring  knowledge  from  retiring  experts  to  their  replacements,  the 
replacements  usually  have  already  acquired  the  necessary  core  knowledge  in  the  subject  area 

under  study.  However,  for  domain  and  personal  knowledge,  stories  may  be  the  most  effective 
way  to  transfer  knowledge  in  a  context. 

Lessons  about  the  nature  of  expertise  from  the  science  of  learning  in  the  last  section 
identified  the  importance  of  having  a  context  for  facts  and  ideas  that  make  up  knowledge. 
Stories  and  narrative  provide  this  context.  Stories  are  a  natural  way  people  share  knowledge  and 
experiences.  Stories  are  more  memorable  than  a  list  of  disassociated  facts  and,  therefore,  more 
likely  to  acted  upon.  Stories  also  tend  to  be  elaborated  by  vivid  images  in  a  learner’s  mind 
drawn  from  past  experiences.  In  this  way,  stories  “tap  into”  a  learner’s  tree  of  talking. 

Havmg  introduced  a  simplified  model  for  knowledge  transfer  and  made  a  case  for  story- 
centered  learning,  it  remains  to  apply  these  concepts  within  the  framework  of  an  organization. 
What  processes  must  be  in  place  to  efficiently  transfer  knowledge  within  an  organization?  Here 
the  focus  changes  from  considering  a  retiring  expert  as  a  sender  and  a  learner  as  a  receiver  to 
considenng  the  context  of  the  entire  organization’s  workforce.  Next  the  discussion  turns  to  the 
processes  that  make  purposeful  knowledge  preservation  occur  broadly  in  an  organization. 


110  Swap,  “Transferring  Tacit  Knowledge  Assets.” 
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Three  Main  Processes  of  Knowledge  Preservation.  Romhardt  identified  three  main 
processes  for  organizational  knowledge  preservation;  selection,  storage,  and  actualization. 
Figure  7  shows  these  processes.  Not  all  knowledge  is  worth  preserving.  Some  knowledge  is 
outdated.  Still  more  knowledge  is  buried  within  a  sea  of  project  reports,  meeting  minutes,  and 
working  files  too  numerous  to  keep  track  of.  The  unexpected  departure  of  a  worker  can  leave 
many  aspects  of  important  projects  poorly  documented.  For  these  reasons,  Romhardt  points  out 
the  need  to  develop  selection  rules  to  aid  in  determining  what  knowledge  to  preserve.  The  goal 
is  to  transfer  valuable  data,  information,  and  skills  into  “organizational  systems”  which  can  be 
used  by  the  whole  organization.  One  way  to  make  selection  rules  work  is  to  assign  clear 
responsibilities  for  the  analysis  and  selection  of  knowledge  to  be  preserved.  Romhardt  points  out 
that  every  organization  has  “centers  of  competence”  from  which  to  assign  a  manager  or  team 
with  the  expertise  to  decide  which  knowledge  is  worth  preserving.  Just  as  information  must  be 


Figure  7 

Three  Main  Knowledge  Preservation  Processes 
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discarded  in  the  tree  of  talking  communication  model  for  an  expert  to  communicate  his 
knowledge,  a  similar  process  must  take  place  at  the  organizational  level  as  knowledge,  in  the 
form  of  single  contributions  and  documents,  must  be  condensed  into  central  documents.  This 
occurs,  for  example,  when  analyses  and  lessons  learned  on  thirteen  missile  tests  are  summarized 
into  one  document.  Users  are  not  forced  to  read  and  interpret  all  the  individual  missile  test 
analyses  to  learn  the  important  aspects  of  the  available  knowledge.1 11 

Romhardt  categorizes  the  storage  of  organizational  knowledge  in  three  forms:  individual , 
collective ,  and  electronic  storage.  Knowledge  stored  in  the  heads  of  individuals  is  where  it  is 
most  useful,  but  also  the  most  volatile.  Retention  incentives  are  one  way  to  keep  valuable 
employees  with  the  organization  and  preserve  individual  knowledge.  However,  retention 
mcentives  generally  do  not  apply  to  retiring  workers.  Flexible  retirement  and  part-time 
employment  are  other  options  to  retain  access  to  valuable  employees  and  their  knowledge. 
When  these  options  are  not  practical,  steps  can  be  taken  to  capture  and  store  knowledge. 
Romhardt  notes  that  structured  exit-interviews  conducted  by  trained  experts  can  be  an  effective 
way  to  capture  critical  knowledge.  He  uses  Sandia  National  Laboratory’s  videotaping  program 
as  a  positive  example.112  Similar  story-based  interviews  with  retiring  analysts  are  part  of  the 
knowledge  preservation  model  developed  in  this  section. 

Knowledge  can  also  be  stored  in  a  collective  entity  such  as  a  group.  Here  limitations  in 
the  ability  of  a  single  individual  to  recall  information  can  be  improved  by  other  members  of  a 
team  or  group.  For  many  fast-moving  project  teams,  the  only  practical  way  to  capture  central 


11 1  Kai  Romhardt,  “Processes  of  Knowledge  Preservation:  Away  from  a  Technology  Dominated 

Approach  <htj://www.dfki.uni-kl.de/~aabecker/Freiberg/Final/Romhardt/ronihardt.html> 

[April,  27,  2002]. 
ni  Ibid. 
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project  status  information  is  in  the  form  of  meeting  minutes.  Romhardt  notes  that  some  Japanese 
companies  emphasize  the  importance  of  recording  good  meeting  minutes  and  provide  special 
training  for  moderators.  Shared  language,  where  words  and  terms  take  on  a  special  meaning 
within  a  group  because  of  shared  experience,  are  also  another  form  of  collective  storage. 

Advances  in  computer  technology  are  making  it  possible  to  store  vast  amounts  of 
information  in  digital  form  at  reasonable  costs.  Here  the  guidance  from  the  previous  section  on 
having  some  but  not  too  much  structure  applies.  Some  degree  of  structure  allows  knowledge  to 
be  found  efficiently.  Some  flexibility  in  structure  allows  the  system  to  accommodate  knowledge 
that  is  difficult  to  categorize  in  precise  terms.  Other  important  concepts  for  electronic  storage 
include  using  a  controlled  vocabulary  and  indexing  schemes  which  are  important  to  allow 
efficient  access  to  knowledge  and  help  prevent  valuable  knowledge  from  being  filed  in  the 
wrong  places  and  potentially  becoming  lost. 

Knowledge  must  be  actualized  or  put  to  use  to  realize  the  benefits  of  knowledge 
preservation.  The  knowledge  being  preserved  needs  to  be  translated  and  applied  in  the  daily 
activities  of  workers.  Otherwise,  knowledge  fades  over  time  and  can  be  lost.  The  major 
problems  Romhardt  cites  here  are  trust  and  access.  Users  must  have  confidence  in  the  accuracy 
and  timeliness  of  knowledge.  A  strategy  here  is  to  assign  responsibility  for  certifying  accuracy 
to  the  centers  of  competency  experts  mentioned  above.113  Having  briefly  defined  three  main 
processes  for  knowledge  preservation,  it  is  time  to  construct  a  model  for  knowledge  preservation 
that  rests  on  the  foundations  that  have  been  laid. 

A  Three-Pronged  Model  for  Knowledge  Preservation.  To  understand  how  the  principles 
discussed  up  to  this  point  can  be  harnessed  to  construct  a  model  for  knowledge  preservation, 

113  'ibid. 
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consider  how  Christensen  describes  an  analogy  dealing  with  the  development  of  heavier-than-air 
manned  flight: 


...  the  ancients  who  attempted  to  fly  by  strapping  feathered  wings  to 
their  arms  and  flapping  with  all  their  might  as  they  leapt  from  high 
places  invariably  failed.  Despite  their  dreams  and  hard  work,  they  were 
fighting  against  some  very  powerful  forces  of  nature.  No  one  could  be 
strong  enough  to  win  this  fight.  Flight  became  possible  only  after 
people  came  to  understand  the  relevant  natural  laws  and  principles  that 
defined  how  the  world  worked:  the  law  of  gravity,  Bernoulli’s  principle, 
and  the  concepts  of  lift,  drag,  and  resistance.  When  people  then 
designed  flying  systems  that  recognized  or  harnessed  the  power  of  these 
laws  and  principles,  rather  than  fighting  them,  they  were  finally  able  to 
fly  to  heights  and  distances  that  were  previously  unimaginable,  4 

Transferring  knowledge  between  people  has  its  own  set  of  relevant  natural  laws  and  principles. 

Two  principles  that  can  be  harnessed  to  design  knowledge  preservation  systems  are  one-on-one 

human  interaction  and  high  bandwidth  communications . 

The  advantages  of  one-on-one  human  interaction  include  the  opportunity  for  feedback 
between  the  retiring  expert  and  the  learner,  opportunities  for  tacit  knowledge  transfer  through 
socialization  and  “getting  to  know  one  another,”  and  interaction  at  levels  higher  up  the  tree  of 
talking.  High  bandwidth  transfer  comes  automatically  with  one-on-one  human  interaction.  If 
you  cannot  have  one-on-one  human  interaction,  it  is  still  possible  to  achieve  relatively  high 
bandwidth  transfer  using  video  or  other  rich  media.  With  a  desire  for  human  interaction  and 

high  bandwidth  in  mind,  the  goal  for  a  knowledge  preservation  model  is  to  maximize  the  use  of 
these  two  features. 


From  this  perspective,  the  constraining  factor  becomes  the  amount  of  time  a  retiring 

expert  has  to  interact  with  learners.  It  takes  time  to  learn  from  an  expert,  in  some  cases,  many 

years.  Many  sources  point  to  traditional  mentoring  as  the  best  way  to  transfer  expertise-related 

”4  Clayton  MXhristensen,  The  Innovator’s  Dilemma:  When  New  Technologies  Cause  Great 
Firms  to  Fail  (Boston,  MA:  Harvard  Business  School  Press,  1997),  pp.  xviii-xix. 
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knowledge  from  an  expert  to  a  learner.115  This  is  not  too  surprising  since  mentoring  consists 
primarily  of  one-on-one  human  interaction.  The  first  step  in  constructing  a  knowledge 
preservation  model  is  to  establish  a  lower  bound  on  how  long  it  takes  to  establish  a  productive 
traditional  mentoring  relationship.  One  can  call  this  the  “ample  time  available”  case.  That 
leaves  two  other  cases:  one  case  where  the  retiring  expert  lacks  enough  time  for  traditional 
mentoring,  but  has  some  time  available  for  one-on-one  human  interaction,  the  “some  time 
available”  case;  and  a  final  case  where  the  retiring  expert  has  little  or  no  time  for  one-on-one 
human  interaction,  the  “limited  time  available”  case. 

Ample  Time  Available  Case.  Some  lower  bounds  on  the  amount  of  time  required  for  a 
traditional  mentoring  relationship  can  be  estimated  in  the  following  way.  First,  consider  that  it 
takes  time  to  get  to  know  someone  at  work.  It  takes  time  to  understand  a  person’s  situation  and 
style  of  approaching  problems.  It  takes  time  to  establish  a  sense  of  trust  and  to  figure  out  the 
boundaries.  Perhaps  this  can  be  accomplished  in  as  little  as  six  months,  but  for  the  purposes  of 
argument  assume  this  process  takes  one  year.  Second,  after  getting  to  know  someone,  it  takes 
more  time  to  work  on  enough  substantive  projects  to  get  exposure  to  the  variety  of  problems 
needed  to  get  the  most  from  an  expert-guided  learning  experience.  Assume  this  takes  an 
additional  two  to  three  years  on  the  job.  That  means  it  takes  three  to  four  years  to  make 
traditional  mentoring  a  viable  option.  Can  meaningful  mentoring  take  place  in  less  time? 
Perhaps,  but  people  still  need  to  get  to  know  each  other  and  they  need  time  to  work  on  a  variety 
of  projects.  This  line  of  reasoning  and  the  research  on  mentoring  suggests  that  the  minimum 
time  necessary  for  a  traditional  mentoring  approach  is  three  years. 


See  Swap  et  al.,  “Transferring  Tacit  Knowledge  Assets”  for  a  discussion  of  mentoring  and  a 
review  of  the  relevant  literature. 
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Some  Time  Available  Case.  If  it  takes  at  least  three  years  to  make  traditional  mentoring 
viable,  what  can  be  done  if  less  time  is  available  for  one-on-one  interaction?  Most  options  will 
likely  result  in  a  tradeoff  between  depth  and  breadth  of  learning.  One  idea  that  has  been 
proposed  is  to  have  a  learner  build  a  case  study  based  on  a  significant  experience  or  project  from 
a  retiring  expert  s  career  and  then  explain  the  case  study  to  a  suitable  audience.116  Building  such 
a  case  study  would  involve  substantive  one-on-one  interaction  between  a  learner  and  a  retiring 
analyst.  A  case  study  serves  to  concentrate  both  parties  on  a  single  series  of  related  issues  to 
make  the  most  of  the  time  available.  Building  such  a  case  study  would  have  an  added  advantage 
of  creating  an  important  artifact  from  which  others  can  leam.  Since  a  learner  starts  with  the 
background  and  experience  he  or  she  brings  to  the  table  and  ends  with  a  finished  case  study, 
whatever  the  level  of  existing  knowledge,  it  will  necessarily  be  engaged.  The  limited  time 
available  creates  a  sense  of  urgency  and  the  learner’s  desire  to  avoid  embarrassment  serves  as 
motivation.  Having  to  explain  the  case  study  also  ensures  that  aspiring  analysts  leam  with 
understanding.  “Explanation  depends,  of  course,  on  understanding:  if  you  do  not  understand 
something,  you  cannot  explain  it.”1 17  It  is  estimated  that  significant  knowledge  transfer  could  be 
accomplished  in  as  little  as  one  or  two  years  with  a  rigorously  structured  case  study  approach. 

Limited  Time  Available  Case.  What  if  there  is  very  limited  time  available  for 
substantive  one-on-one  interaction?  One  idea  is  to  have  the  retiring  expert  write  up  his  or  her 
expenences  m  a  document  or  memoir.  Unfortunately,  this  is  not  a  high  bandwidth  option  and  the 
ease  with  which  one  can  leam  from  such  a  document  depends  on  the  expert’s  skill  as  a  writer. 
Another  idea  is  to  videotape  interviews  or  panel  discussions  with  experts  or  panels  of  experts. 

xt  jdea  originated  independently  in  discussions  with  Professor  Jim  Fitzsimonds  of  the 
Naval  War  College  and  in  interviews  conducted  with  intelligence  analysts  at  ONI 
Johnson-Laird,  Mental  Models,  p.  2. 
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Videotaping  efforts  to  capture  knowledge  at  DOE  and  NAVAIR  were  discussed  in  the  previous 
section.  In  the  more  sophisticated  forms  of  this  approach,  the  videos  are  digitized  and  transcripts 
are  created  to  allow  searchable  access  over  a  computer  network. 

The  combination  of  these  three  approaches:  traditional  mentoring ,  learner-built  case 
studies,  and  digital  video  archiving  can  be  viewed  as  a  three-pronged  model  for  knowledge 
preservation.  Figure  8  shows  the  relationships  between  experts,  learners,  knowledge,  and  stories. 
Learner-built  case  studies  can  be  viewed  as  a  “hand-off’  approach  because  they  make  use  of 


Figure  8 

Three-Pronged  Model  for  Knowledge  Preservation 
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one-on-one  interaction  between  a  retiring  expert  and  a  learner.  The  digital  video  archiving  can 
be  viewed  as  an  “over-the-wall”  approach  because  it  can  be  used  by  learners  without  having  one- 
on-one  interaction  with  a  retiring  expert. 
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The  time  frames  suggested  above  are  notional,  and  some  experts  may  use  all  three 
approaches  before  they  retire.  For  example,  an  expert  who  is  planning  to  retire  in  five  years  may 
enter  into  a  mentoring  relationship  with  a  learner  today.  Two  and  a  half  years  later  the  expert 
may  work  with  another  learner  to  build  a  case  study  and,  finally,  before  the  expert  retires,  digital 
videos  may  by  created  to  capture  knowledge  using  this  approach  as  well.  Having  introduced  a 
three-pronged  model  it  is  appropriate  to  consider  what  is  involved  with  each  prong. 

Traditional  Mentoring.  Traditional  mentoring  can  be  formal  or  informal.  In  other 
words,  the  mentor  and  mentee  can  be  paired  together  and  enter  into  formal  written  mentoring 
agreement  or  they  can  seek  each  other  out  and  begin  a  mentoring  relationship  without  a  written 
agreement.  Either  way,  mentors  act  as  informal  teachers  and  coaches  to  transfer  knowledge. 
The  knowledge  transferred  between  a  mentor  and  mentee  can  be  in  the  form  of  feedback  or 
guidance  on  how  to  solve  problems.  Problem  solving  guidance  is  usually  provided  only  after  the 
mentee  has  struggled  for  some  time  with  a  problem.  Advice  on  how  to  maneuver  and  function  in 
the  bureaucracy  is  another  form  of  knowledge  mentors  provide  to  mentees.  Other  than  direct 
experience,  this  can  be  the  only  way  to  acquire  this  kind  of  knowledge.118 

Swap  et  al.,  note  that  mentors  use  various  teaching  techniques  but  most  can  be 
characterized  using  two  dimensions.  The  first  is  experience.  This  includes  a  mentee’ s  personal 
experience,  often  guided  by  the  mentor,  and  vicarious  experience  or  the  experiences  of  others. 
Vicarious  experiences  can  be  the  mentor’s  experiences  or  the  experiences  of  others  that  the 
mentor  relates  to  the  mentee.  The  second  dimension  is  direction  in  teaching.  This  can  take  the 


118 

Swap,  “Transferring  Tacit  Knowledge  Assets.” 
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form  of  either  self-directed  teaching  or  instruction  directed  by  the  mentor.  119  These 
mentor/mentee  interactions  are  shown  in  figure  9. 


Figure  9 
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There  are  many  examples  of  mentoring  programs  to  draw  ideas  from  to  establish  a 
traditional  mentoring  program.  The  Department  of  the  Navy  Human  Resources  Office  has  a  web 
site  and  has  developed  a  mentoring  handbook.120  The  National  Aeronautics  and  Space 
Administration’s  (NASA)  Goddard  Space  Flight  Center  uses  mentoring  to  develop  project 


119  Ibid. 

1 20 

The  Department  of  the  Navy  Human  Resources  web  site  and  mentoring  handbook  are  geared 
toward  leadership  development.  However,  many  of  the  same  principles  apply  to  mentoring  to 
transfer  expertise.  <http://www.donhr.navy.mil/Employees/cld.asp>  and 
<http://www.donhr.navy.mil/managers/menthb.pdf>  [May  20, 2002], 
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managers  and  technical  staff  and  their  web  site  does  a  good  job  explaining  the  relationship 
between  and  responsibilities  of  mentor,  mentee,  and  the  mentee’s  supervisor.121 

Learner-Built  Case  Studies.  Having  a  learner  build  and  explain  one  or  more  case  studies 
based  on  the  experiences  of  a  retiring  expert  is  an  approach  intended  to  accomplish  several 
things.  First,  it  makes  the  most  of  the  limited  time  available  by  focusing  the  learner  and  the 
retiring  expert  on  one  specific  example  that  is  treated  in  depth.  Second,  the  learner  and  the 
retiring  expert  work  together  to  document  one  case  from  the  expert’s  background.  This  involves 
purposeful  one-on-one  interaction  which  is  a  goal  from  the  knowledge  transfer  model  (figures  5 
and  6).  Third,  whatever  the  level  of  a  learner’s  existing  knowledge  and  experience,  it  will  be 
engaged  as  the  starting  point  leading  to  the  finished  case  study.  Finally,  the  end  product  will 

serve  to  capture  and  document  important  knowledge  from  the  retiring  expert  that  can  be  used  by 
other  learners. 

There  are  many  examples  of  case  studies.  This  is  a  popular  teaching  technique  at 
business  schools,  and  the  Naval  War  College  uses  case  studies  in  its  Policy  Making  and 
Implementation  course.  The  Central  Intelligence  Agency  (CIA)  has  developed  several  case 
studies  in  intelligence  that  can  also  be  used  as  models.123 

The  retiring  expert  should  identify  the  case  study  for  the  learner  to  build.  It  is  suggested 
that  future  relevance  and  the  availability  of  supporting  materials  (including  mental  recollections 
and  documents)  be  used  in  identifying  candidate  cases  for  study.  Interview  techniques  such  as 


121 

]22  See  <http://fpd.gsfc.nasa.gov/cd/mentors.html>  [May  16,  2002]. 

Valerie  J.  Lofland  and  David  A.  Williams  eds..  Case  Studies  in  Policy  Making  and 
Implementation  5  Edition  (Newport,  RI:  The  Naval  War  College,  2001) 

CIA  Case  Studies  in  Intelligence  are  available  over  Intel  link  at 
<http://www.csi.cia.ic.gov/studies/vol41nol/index.html>  [April  3,  2002] 
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cognitive  task  analysis  to  elicit  knowledge  can  also  be  used  to  help  build  the  case  study.124  The 
use  of  narrative  or  story-based  construction  is  recommended  for  the  reasons  stated  previously. 

Digital  Video  Interview  and  Story  Archive.  In  its  simplest  form,  this  method  of 
knowledge  capture  consists  of  video  taping  sessions  with  an  expert  or  panel  of  experts  as  they 
discuss  stories  of  how  they  solved  difficult  or  puzzling  problems,  how  they  were  surprised  by 
unexpected  events,  and  how  they  used  innovative  techniques  to  improve  performance.  The 
resulting  library  of  video  tapes  are  not  the  easiest  form  for  learners  to  access.  It  is  not  possible 
for  learners  to  easily  search  video  tapes,  and  they  often  have  to  go  to  a  central  location  to  view 
them.  To  overcome  these  handicaps,  products  have  been  created  to  digitize  the  video  so  learners 
can  search  and  view  the  interviews  over  a  computer  network  at  their  desks.  I  call  this  third 
prong  of  the  three-pronged  model  the  “Over-the-Wall”  approach  of  knowledge  preservation 
because  it  can  be  used  without  one-on-one  interaction  between  a  learner  and  a  retiring  analyst. 

Producing  videos  of  retiring  experts  telling  stories  and  being  interviewed  is  an  approach 
that  provides  a  relatively  high  bandwidth  learning  channel  when  one-on-one  interaction  between 
experts  and  learners  is  not  possible.  Selecting  appropriate  stories  and  indexing  them  are  key 
ideas  to  make  the  videos  more  useful  as  learning  tools. 

Again,  the  retiring  expert  will  need  to  help  identify  the  stories  to  record.  It  is  suggested 
that  future  relevance  be  used  as  a  guide  in  selecting  stories.  Interview  techniques  such  as 
cognitive  task  analysis  to  elicit  knowledge  can  also  be  used  to  identify  stories  and  important 
points  to  emphasize.125  Indices  must  then  be  created  to  assist  in  on-line  retrieval.  One  way  to 

124  Information  on  interview  techniques  can  be  found  at  the  CTA  Resource  web  site 
<http://www.ctaresource.com>  [January  15,  2002]  and  the  Navy’s  Office  of  Training 
Technology  <http://www.ott.navy.mil>  [January  15, 2002] 

125  Ibid. 
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identify  indices  is  by  conducting  a  knowledge  audit  to  identify  specific  examples  from  categories 
such  as  diagnosing  and  predicting,  perceptual  skills,  tricks  of  the  trade  and  when  to  apply  them, 
improvising,  and  recognizing  anomalies.126  These  categories  are  given  as  examples.  There 
undoubtedly  other  equally  valid  categories  that  can  be  used.  These  examples  can  serve  as 
candidate  indices  that  the  retiring  expert  can  use  to  think  of  stories  that  serve  to  illustrate 
expertise  related  to  the  categories  and  to  ensure  these  points  are  captured  in  stories. 

Making  digitized  video  clips  available  over  a  computer  network  makes  the  knowledge 
easier  to  access  and  allows  specific  topics  to  be  viewed  without  sitting  through  an  entire  tape. 
There  are  products  on  the  market  designed  to  produce  and  distribute  this  type  of  digital  video.127 
NASA’s  Goddard  Space  Flight  Center  uses  story-based  digital  video  clips  to  preserve  lessons 
learned  from  various  project  managers.  Readers  can  get  a  flavor  for  this  approach  by  sampling 

some  video  clips  of  Jerry  Madden,  a  retired  NASA  project  manager,  telling  stories  and  relating 
knowledge  from  his  career.128 


126  Laura  G.  Militello 
Practitioner’s  Toolkit 
1618-1641. 


and  Robert  J.  B.  Hutton,  “Applied  Cognitive  Task  Analysis  (ACTA):  A 
for  Understanding  Cognitive  Task  Demands,”  Ergonomics  41  (1998): 


1 27 

128  See  <httP://www-convera.com>  [April  23, 2002], 

See  <http://fpd.gsfc.nasa.gov/cd/video.html>  [  May  16, 2002]. 
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Summary.  Steps  for  each  approach  in  the  three-pronged  model  are  summarized  in  table 
2.  The  time  frames  overlap  and  should  not  be  taken  as  hard  and  fast  values.  There  is  flexibility 
for  an  organization  to  employ  these  approaches  based  on  the  nature  of  the  expertise  involved,  the 
particular  retiring  experts  and  learners,  and  experience  and  lessons  to  be  learned  using  these 
approaches.  Both  the  over-the-wall  and  hand-off  approaches  require  some  up  front  work  to 


Table  2 

Steps  in  Three-Pronged  Model 


Time  Available 

Less  than 

1  year 

1  to  3  years 

More  than 

3  years 

Name 

"Over-the-Wall" 

“Hand-Off” 

traditional 

Mentoring 

Steps 

•  Identify  key 
indices 

•  Identify 
interesting 
stories 

•  Make  videos 

•  Create  digital 
archive 

•  Learners  use 

•  Identify  key 
indices 

•  Identify 
interesting 
stories 

•  Learner  builds 
case  study 

•  Learner 
presents 

•  Identify 
mentor- 
proteges 

•  Self  vs 
directed 
teaching 

•  Personal  vs 
vicarious 
experience 

identify  the  indices  that  are  key  to  how  the  expert’s  knowledge  is  organized  and  used.  Also, 
stories  with  the  most  interesting  and  likely-to-be-relevant  knowledge  need  to  be  identified  in  the 
first  step. 


In  this  section,  a  generalized  model  of  knowledge  preservation  was  developed  building 
on  a  simplified  model  of  knowledge  transfer  derived  from  the  concepts  of  logical  depth  and 
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exformation.  A  case  was  made  to  adopt  a  story-based  approach  to  learning.  These  ideas  were 
combined  to  arrive  at  a  three-pronged  model  based  on  how  long  retiring  experts  are  available  to 
work  face-to-face  with  their  replacements. 

Although  the  focus  here  has  been  on  the  transfer  of  knowledge  between  people,  one 
should  not  discount  the  value  of  an  archiving  effort  for  data  and  information.  Archiving  makes 
the  artifacts  of  technical  knowledge  more  readily  available.  Many  of  the  lessons  learned  in  using 
the  three-pronged  model  will  point  toward  the  data  and  information  that  properly  belongs  in  an 
archive.  However,  an  archive  by  itself  will  not  serve  to  transfer  knowledge  from  one  head  to 
another.  In  the  next  section  the  issues  of  practicality  that  must  be  considered  to  put  this 
knowledge  preservation  model  into  action  are  discussed. 
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SECTION  VI 


PUTTING  THE  MODEL  INTO  ACTION 

Knowledge  Superiority  based  on  Maritime  Intelligence ... 

...  Enabling  Dominance  from  the  Sea. 

--Inscription  at  ONI  Headquarters 

Introduction.  In  this  section  the  discussion  turns  to  how  to  put  the  three-pronged 
knowledge  preservation  model  into  action  at  the  Office  of  Naval  Intelligence  (ONI).  The 
objective  is  to  frame  specific  answers  to  the  questions  raised  in  section  III.  What  constitutes 
expertise  in  conducting  Naval  intelligence  analysis  and  how  does  one  identify  and  prioritize 
expertise  worth  preserving?  How  can  expertise-related  knowledge  be  captured  in  a  form  that 
fosters  learning  and  transfer?  What  characteristics  do  potential  solutions  need  to  possess  to  be 
seen  as  practical  by  analysts  and  senior  Navy  management?  The  answers  to  these  questions  will 
be  framed  around  the  processes  and  model  explained  in  the  previous  section.  The  lessons  from 
industry,  science,  and  government  will  also  be  used  to  guide  the  following  discussion.  Some  ad 
hoc  knowledge  preservation  initiatives,  already  taking  place  at  ONI,  are  discussed,  and  some 
ideas  for  how  these  initiatives  fit  into  the  model-based  solution  are  presented.  The  thorny  issues 
of  cost,  schedule,  and  culture  are  briefly  treated  and  some  Rough  Order  of  Magnitude  (ROM) 
cost  estimates  are  presented  along  with  their  underlying  assumptions. 

Characteristics  of  Expertise  in  Naval  Intelligence.  To  select  specific  expertise-related 
knowledge  for  preservation,  it  must  first  be  identified.  Are  there  characteristics  of  expertise  in 
Naval  intelligence  that  are  shared  with  expertise  in  other  fields?  Researchers  have  identified 
several  “hallmarks”  of  expertise  that  apply  to  expertise  in  general  and  have  implications  that  are 
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useful  for  identifying  expertise-related  knowledge  and  capturing  knowledge  that  includes 
elements  that  benefit  learners.129  The  hallmarks  of  expertise  are: 

•  Experts  notice  features  and  meaningful  patterns  that  are  not 
noticed  by  novices. 

•  Experts  have  acquired  a  great  deal  of  content  knowledge  that  is 
organized  in  ways  that  reflect  a  deep  understanding  of  their 
subject  matter. 

•  Experts’  knowledge  cannot  be  reduced  to  sets  of  isolated  facts  or 
propositions  but,  instead,  reflects  contexts  of  applicability:  that 
is,  the  knowledge  is  “conditionalized”  on  a  set  of  circumstances. 

•  Experts  are  able  to  flexibly  retrieve  important  aspects  of  their 
knowledge  with  little  attentional  effort. 

•  Though  experts  know  their  disciplines  thoroughly,  this  does  not 
guarantee  that  they  are  able  to  teach  others. 

•  Experts  have  varying  levels  of  flexibility  in  their  approach  to 
new  situations.130 

It  is  reasonable  to  assume  that  these  hallmarks  apply  to  expertise  in  Naval  intelligence. 
Specific  characteristics  of  expertise  in  Naval  intelligence  analysis  are  tied  to  each  subspecialty. 
For  example,  these  specifics  include  the  features  and  patterns  a  senior  submarine  analyst  might 
notice  in  raw  source  data  that  a  novice  would  not  notice.  This  includes  specific  details  of  the 
contexts  under  which  certain  rules  of  thumb  or  analytical  methods  apply.  These  generalized 
hallmarks  and  the  specific  details  that  emerge  when  they  are  applied  to  the  subspecialties  within 
Naval  intelligence  constitute  the  characteristics  of  expertise  in  conducting  Naval  intelligence 
analysis. 

Equipped  with  some  idea  of  what  constitutes  expertise  in  Naval  intelligence  analysis, 
how  does  one  go  about  selecting  expertise-related  knowledge  worth  preserving?  In  other  words, 
how  does  one  identify  and  prioritize  expertise? 


129  Bransford  et  al.,  How  People  Learn,  p.  31. 

130  Ibid.,  p.  31. 


78 


Identifying  and  Prioritizing  Expertise:  Selection.  Preserving  knowledge  costs  time  and 
money.  Therefore,  the  focus  needs  to  be  on  preserving  knowledge  that  will  be  useful  in  the 
future.  The  question  then  becomes  how  to  make  future  relevance  the  driving  consideration  of  the 
knowledge  to  be  preserved.  An  agency  or  organization’s  strategic  planning  efforts  and  strategic 
plans  provide  one  source  of  high  level  guidance.  Although  ONI  does  not  have  a  current  strategic 
plan,  Joint  Vision  2020  and  the  Navy  Strategic  Planning  Guidance  can  offer  some  broad  areas  to 
act  as  guidance.  Other  approaches  that  can  be  used  to  get  a  handle  on  future  relevance  at  the 
organizational  level  include  scenario  planning  techniques  and  capability-based  approaches  like 
the  Design  Reference  Mission.  Yet  another  approach  for  understanding  the  future  relevance 
of  knowledge  is  to  adapt  a  threat  projection  methodology  used  to  estimate  the  timeline  and 
numbers  associated  with  the  fielding  of  an  enemy  weapons  system  to  focus  on  future  Naval 
intelligence  capabilities.  A  candidate  threat  projection  methodology  to  adapt  for  this  purpose  is 


Joint  Vision  2020  is  the  Department  of  Defense’s  strategic  vision  for  the  future.  It  was 
written  by  the  Chairman  of  the  Joint  Chiefs  of  Staff.  Each  Service  has  developed  a  companion 
vision  document  as  well.  These  documents  can  be  accessed  at  <http://www.dtic.mil/jv2020/> 
[May  28,  2002].  See  Chief  of  Naval  Operations,  Navy  Strategic  Planning  Guidance  with  Long 
Range  Planning  Objectives,  April  2000.  <http://www.hq.navy.mil/n3n5/files/NSPG2000  pdf> 
[May  28,  2002], 

For  an  example  of  a  scenario  technique  that  uses  vision  stories  to  better  understand  the  future 
see  Ira  M.  Levin,  “Vision  Revisited:  Telling  the  Story  of  the  Future,”  The  Journal  of  Applied 
Behavioral  Science  36  (2000):  91-107.  The  Design  Reference  Mission  (DRM)  concept 
originates  from  the  Systems  Engineering  community.  DRM  has  been  used  by  the  Navy  to 
understand  future  force  requirements  at  the  battleforce  and  platform  levels.  DRM  is  aimed  at 
characterizing  the  future  threat  and  operating  environment  for  use  with  requirements  definition 
concept  development,  and  trade  study  analysis.  Since  a  number  of  DRMs  have  been  created  for 
Naval  warfare  areas,  these  same  products  could  be  used  to  gauge  future  Naval  intelligence 
requirements,  capabilities,  and  the  potential  relevance  of  knowledge.  See  Fred  R.  Skolnick  and 
Phillip  G.  Wilkins,  “Laying  the  Foundation  for  Successful  Systems  Engineering,”  Johns  Hopkins 
APL  Technical  Digest  21  (2000):  208-216  and  Christine  Salamacha,  Sterling  Smoot,  and 
Kathleen  Farris,  “C4ISRT  in  an  Operational  Context,”  Johns  Hopkins  APL  Technical  Digest  21 
(2000):  378-386. 
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one  developed  by  the  Naval  Postgraduate  School  which  uses  the  past  as  a  key  to  the  future.133 
The  strength  of  this  type  of  methodology  lies  not  in  its  accuracy  in  predicting  the  future,  but  in 
the  value  of  having  the  rationale  and  reasoning  behind  the  projections  documented  so  that 
unanticipated  changes  can  be  incorporated  and  independent  scrutiny  of  the  projections  can  be 
made.  These  features  make  it  well  suited  for  estimating  the  analysis  capabilities  needed  in  the 
future  and  the  potential  relevance  of  expertise-related  knowledge  being  considered  for  inclusion 
in  a  knowledge  preservation  program. 

The  discussion  of  knowledge  preservation  processes  in  the  previous  section  suggested 
creating  “selection  rules”  to  determine  knowledge  worth  preserving.  Such  a  set  of  rules  could  be 
created  for  ONI.  One  rule  might  be  to  perform  periodic  Naval  intelligence  analysis  capability 
assessments  for  each  critical  area  of  interest  using  some  combination  of  the  methods  discussed 
above.  Example  areas  of  interest  include  Naval  guns,  directed  energy  weapons,  submarines, 
surface  ships,  Naval  Surface  to  Air  Missiles  (SAMs),  and  cruise  missiles.  The  frequency  of  the 
assessments  could  vary  depending  on  the  technologies  involved,  but  could  be  done  once  every 
five  years  for  example. 

Further  selection  rules  could  be  developed  from  the  results  of  these  capability 
assessments  to  pinpoint  specific  analytical  expertise.  Selection  rules  must  be  flexible,  since  it  is 
difficult  to  anticipate  all  areas  of  knowledge  that  will  be  needed  in  the  future.  Selection  rules  and 


133  D.  K.  Abernathy,  “Threat  Projections  -  What  You  Don’t  Know  Can  Hurt  You.”  Unpublished 
Paper,  Naval  War  College,  Newport,  RI,  28  October  1982,  quoting  Dr.  R.  Stolfi  and  Dr.  P. 
Wand,  “A  General  Methodology  for  Forecasting  the  Technological  Threat  from  the  Soviet 
Navy,”  instructional  material  provided  to  the  author  during  a  class  held  at  the  Naval  Intelligence 
Support  Center,  September  27-28, 1976. 


80 


approaches  for  estimating  the  future  relevance  of  knowledge  are  useful  tools,  but  people  must  be 
tasked  to  use  these  tools  for  appropriate  knowledge  to  be  selected  for  preservation. 

Assigning  roles  and  responsibilities  for  knowledge  preservation  to  knowledgeable 
individuals  from  “centers  of  competency”  in  an  organization  is  an  idea  that  should  be  adopted  by 
ONI.  Fortunately,  just  such  a  group  of  knowledgeable  individuals  have  been  formed  into  a  team 
to  study  knowledge  preservation  and  similar  issues.  They  are  the  so  called  “Subject  Matter 
Experts”  (SME)  group  at  ONI.  These  are  the  experts  who  are  in  the  best  position  to  judge  the 
expertise  that  is  most  likely  to  be  relevant  in  the  future.  To  select  knowledge  worth  preserving 
one  should  “ask  the  experts.” 

Asking  the  experts  is  a  sensible  way  to  identify  the  most  obvious  expertise  worth 
preserving,  but  what  about  less  obvious  expertise?  A  strategy  that  can  be  used  here  is  to  look  for 
the  hallmarks  of  expertise  listed  above.  In  some  cases,  this  level  of  expertise  may  exist  in 
analysts  who  are  far  from  retirement.  Preserving  some  of  their  knowledge  today  may  be 
worthwhile  in  the  event  they  leave  the  organization  or  take  another  assignment.  Also,  the 
systematic  preservation  of  this  type  of  expertise  as  it  is  identified  may  be  easier  than  waiting 
until  later  and  may  lead  to  the  sharing  of  best  practices  in  other  areas  of  intelligence  analysis. 

Capturing  Knowledge  in  a  Form  That  Fosters  Learning  and  Transfer:  Storage.  Once 
Knowledge  has  been  selected  to  be  preserved,  it  must  be  captured  in  a  form  that  fosters  learning 
and  transfer.  The  knowledge  must  be  stored.  This  can  be  accomplished  in  a  number  of  different 
ways  but  the  discussion  here  focuses  on  learner-built  case  studies  and  video  interviews,  both  of 
which  use  stories  to  capture  and  transfer  knowledge. 

Planning  and  pre-capture  interview  techniques  are  recommended  to  identify  relevant 
stories  and  to  ensure  the  knowledge  captured  includes  contextual  information,  cues  for 
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metacognition,  tips  on  how  organize  the  taiow.edge,  and  key  indices  to  can  be  used  for 
search  and  retrieval. 


Interview  techniques  based  on  Cognitive  Task  Analysis  (CTA)  methodologies  have  been 
used  to  elicit  expertise-related  knowledge  in  a  number  of  different  settings.'34  One  approach  is 
die  Critical  Decision  Method  (COM)  which  uses  a  retrospective  interview  strategy  and  a  series 


of  cognitive  probe  questions  to  explore  different 


aspects  of  the  decision-making  process.  This 


technique  has  been  used  elicit  expertise-related  knowledge  from  fire  ground  commmders, 
neonatal  intensive  care  uni,  nurses,  in  Army  command  and  control  settings,  and  military  ground 
navigators.135  Another  approach  is  to  adopt  the  Applied  Cognitive  Task  Analysis  (ACTA) 
methodology  developed  by  Militello  and  Button'35  ACTA  is  a  simplified  methodology  that 
makes  use  of  stmcnired  activities  an  interviewer  and  an  expert  work  through  to  elicit  knowledge. 
One  activity  is  the  creation  of  task  diagrams  where  an  expert’s  approach  to  a  task  is  subdivided 
into  between  3  and  6  steps  to  provide  a  manageable,  ye,  meaningful,  level  of  detail  to  begin 
understanding  how  the  expert  conducts  an  important  task.  Another  activity  is  the  knowledge 


S1°”  Makmg  Conftrence’  Tammsvik,  SwSen,  May  2MSL2000. 

135  MTt  11  ■m?yfsmst'tuteoty/|Iarken./publications/ndm5.pdf>  [May  28  20021 
164“  “d  Hu"°n'  ““  C°^  Task  Analysis/’  Ergonomics  4^998):  1618- 


audit  where  probe  questions  are  used  to  explore  important  expertise-related  aspects  of  knowledge 

including  “diagnosing  and  predicting,  situation  awareness,  perceptual  skills,  developing  and 

knowing  when  to  apply  tricks  of  the  trade,  improvising,  metacognition,  recognizing  anomalies, 

1  *1*1 

and  compensating  for  equipment  limitations.” 

Lessons  from  the  science  of  learning  discussed  in  section  IV  suggest  that  these  pre¬ 
knowledge  capture  activities  and  interviews  should  be  structured  to  obtain  a  map  of  the  factual 
knowledge  required,  details  on  the  contexts  in  which  the  knowledge  applies,  and  cues  for 
assessing  a  learner’s  progress  toward  understanding. 

Case  Studies.  In  the  learner-built  case  studies,  a  learner  works  one-on-one  with  a 
retiring  senior  analyst  to  document  a  significant  case  study  from  the  senior  analyst’s  background 
and  career.  The  learner  will  document  the  case  study  and  then  explain  it  to  a  suitable  audience. 
Although  the  best  case  study  to  be  captured  may  already  be  apparent  to  the  retiring  analyst,  the 
interview  techniques  discussed  above  should  still  be  used  to  bring  out  important  details.  And 
when  a  retiring  expert  has  trouble  selecting  a  case  for  study,  the  interview  techniques  can  be  used 
to  help  identify  a  suitable  case. 

Once  key  rules  of  thumb,  tricks  of  the  trade,  and  contextual  knowledge  have  been 
identified  in  the  planning  and  pre-capture  interview  activities,  these  elements  should  be  weaved 
into  the  story  line  as  the  case  is  constructed.  Where  including  these  elements  directly  in  the 
narrative  becomes  cumbersome,  they  should  be  included  as  appendices  and  cross  referenced  so 
learners  can  gain  the  benefit  of  learning  from  a  story  and  have  access  to  critical  supporting 
information  for  further  study. 


137  Ibid. 
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Having  learners  explain  their  case  studies  provides  incentives  for  them  to  leam  with 
understanding.  These  incentives  include  creating  a  sense  of  urgency  and  a  desire  to  avoid 
embarrassment.  The  existing  practice  at  ONI  where  analysts  present  a  “vignette”  of  how  a 
foreign  weapon  system  works  to  the  admiral  could  be  used  as  a  forum  for  learners  to  explain  the 
case  studies  they  have  developed.138  An  additional  benefit  form  the  learner-built  case  study  is 
that  it  documents  this  knowledge  and  makes  it  available  for  others  to  read  and  study  in  the  future. 

Video  Interviews.  Using  video  to  capture  knowledge  from  interviews  with  retiring 
analysts  is  one  way  to  transfer  their  knowledge  without  direct  one-on-one  interaction  between 
the  learners  and  retiring  analysts.  The  planning  and  pre-capture  interview  techniques  described 
above  also  apply  here.  These  activities  can  help  select  the  stories  to  record  and  identify  the  key 
indices,  contextual  knowledge,  and  metacognition  cues  learners  need  to  more  easily  grasp  the 
material  and  reach  the  deep  level  of  understanding. 

Once  the  stories  and  key  information  have  been  identified,  storytelling-interview  sessions 
can  be  videotaped.  The  stories  can  be  told  ad  lib  or  from  a  script,  depending  on  what  makes  the 
retiring  analyst  more  comfortable.  When  a  script  is  not  used,  an  interviewer  can  use  probe 
questions  to  prompt  the  retiring  analyst  to  elaborate  on  key  points  that  do  not  get  covered  in  the 
ad  lib  portions  of  a  storytelling  session. 

Video  from  the  sessions  can  then  be  edited  to  clean  up  miscues  and  retakes  from  the 
storytelling-interview  sessions.  One  senior  ONI  analyst  expressed  concerns  about  the  ability  of 
the  retiring  analyst  to  edit  and  reflect  upon  the  end  product  from  the  video  sessions.139  However, 
these  concerns  can  be  addressed  by  involving  senior  analysts  in  the  editing  and  post-production 


138 

ONI  representatives,  Interview  by  author, 

139  Ibid. 


Suitland,  MD,  3  April  2002. 
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activities.  Although  the  knowledge  capture  team  should  have  training  and  experience  with  video 
editing,  the  senior  analyst,  as  the  source  of  the  knowledge  being  captured,  is  in  the  best  position 
to  guide  the  editing  process.  In  some  cases,  involving  a  senior  analyst  in  a  reflective  editing 
process  may  lead  to  further  video  recording  sessions  to  improve  segments  or  capture  new  aspects 
of  the  stories. 

If  a  digital  video  archiving  and  retrieval  system  is  used,  the  videos  are  digitized  and 
software  is  used  to  create  transcripts  that  can  be  searched  on-line  to  select  content  for  viewing. 
Of  course,  security  and  access  controls  must  be  used  and  should  be  thoroughly  scrutinized  for 
potential  vulnerabilities. 

Several  learning  possibilities  exist  once  the  videos  become  available.  Learners  can  use 
the  video  stories  as  part  of  a  program  of  self-study  or  the  videos  can  be  incorporated  into  more 
formalized  instructional  classes.  Because  of  the  indexing  and  digital  access,  learners  can  enter 
the  video  stories  based  on  an  index  related  cue  such  as  a  “rule  of  thumb”  or  the  videos  can  be 
viewed  in  their  entirety.  This  degree  of  flexibility  suggests  that  analysts  should  experiment  with 
these  capabilities  to  explore  what  works  best  for  them. 

Study  Initial  Learners  to  Develop  Pedagogical  Content  Knowledge.  In  addition  to 
analysts  experimenting  with  video  capabilities,  efforts  should  be  made  to  study  initial  learners  to 
identify  the  roadblocks  and  difficulties  they  encounter  in  understanding  the  knowledge  under 
study.  This  pedagogical  content  knowledge  can  then  be  used  to  make  improvements  in  the 
learning  materials  and  develop  aids  such  as  computer  simulations  to  help  learners  overcome 
barriers  to  learning  and  understanding.  Developing  pedagogical  content  knowledge  requires 
effort  and  resources.  Therefore,  only  knowledge  that  will  be  used  by  several  different  learners 
justifies  this  level  of  investment. 
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Archiving  of  Documents,  Files,  and  Records.  Although  archiving  of  documents,  files, 
and  records  is  not  seen  as  the  principal  putpose  of  the  three-pronged  model  developed  in  this 
dy,  archiving  reinforces  and  complements  knowledge  preservation.  The  stories  and 
knowledge  captured  in  case  studies  and  video  will  often  reference  documents,  files,  and  records. 
These  artifacts  provide  the  source  materials  that  learners  can  use  to  more  fully  explore  aspects  of 
stories  and  to  gain  ideas  for  how  to  adapt  the  knowledge  learned  to  their  own  projects. 

One  senior  analyst  interviewed  by  the  author  explained  that  each  community  of  analysts 
at  ONT  has  developed  its  own  system  for  storing  and  filing  data  and  infoimation  related  to  their 
projects.140  This  often  results  in  working  materials  and  project  files  being  stored  in  desk  drawers 
and  nearby  filing  cabinets.  A  consequence  of  these  practices  is  that  analysts  are  limited  in  their 
ability  to  efficiently  conduct  searches  for  specific  information  and  to  take  advantage  of  data- 
minmg  technology.  Some  knowledge  becomes  essentially  inaccessible  while  an  analyst  is  out  of 
the  office  on  vacation.  A  digitized,  on-line  solution  could  make  this  knowledge  more  accessible 

and  lead  to  improved  analytical  capability  through  increased  sharing  of  ideas  and  information. 
This  same  senior  analyst  said: 


In  the  surface  community,  they  have  50  filing  cabinets  in  a  central 
location  that  are  organized  in  a  logical  manner.  This  has  paid  off  many 
imes  m  the  past  because  of  the  ease  of  finding  what  you  are  looking  for 
due  to  the  organization  and  accessibility.  An  electronic  version  of 
something  like  the  surface  community  filing  cabinets  would  go  a  long 
way  toward  fixing  these  shortcomings. 141  B  S 


This  suggests  that  the  surface  community’s  filing  cabinets  could  be  studied  as  a  model  to 
develop  an  electronic  knowledge  management  solution  for  the  analysts  at  ONI.  One  way  to 
achieve  this  is  by  fielding  a  pilot  project  with  heavy  analyst  involvement  in  the  prototyping  stage 
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to  ensure  the  ease  of  use,  level  of  structure,  and  degree  of  flexibility  required  for  success  are 
designed  into  the  solution  before  committing  the  larger  investment  required  to  put  this 
knowledge  on-line. 

Adopting  a  Process  Orientation  Towards  Intelligence  Analysis.  One  of  the  guidelines 
for  successful  knowledge  management  projects  discussed  in  section  IV  (see  pages  26,  27)  was  to 
adopt  a  process  orientation.  Intelligence  analysis  is  a  knowledge  intensive  endeavor,  so  it  is  not 
surprising  to  see  a  process  perspective  being  advocated  to  improve  the  overall  quality  of 
intelligence  analysis  and  serve  as  a  template  that  analysts  can  use  to  guide  their  own  efforts. 

For  these  reasons,  another  senior  ONI  analyst  is  conducting  an  “analysis  of  analytical 
tradecraft”  with  an  eye  towards  identifying  a  “critical  path”  for  analysts  to  follow.  This  senior 
analyst  is  establishing  some  “corporate  standards  on  how  analysis  is  done”  as  the  first  step  in 
defining  a  process  of  analytical  tradecraft.142  For  example,  this  might  include  quantifying  and 
validating  fifteen  or  so  common  attributes  a  good  analytical  process  must  contain.  The 
development  of  standards  are  followed  by  a  more  detailed  “process  of  analytical  tradecraft.”  The 
logic  behind  these  process  initiatives  is  that  by  following  a  sound  process,  analysts  should  be  led 
to  the  right  answers  more  often  than  not.  Following  a  sound  process  could  also  lead  to  the 
adoption  and  sharing  of  analytical  best  practices  by  more  analysts,  and  it  could  result  in  greater 
consistency  in  how  analysis  is  documented.  Better  documented  analysis  should  result  in  analysis 
that  better  stands  up  to  scrutiny  and  is  easier  for  fellow  analysts  to  understand  and  verify. 


142  Ibid. 
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Ad  Hoc  Initiatives.  Several  ad  hoc  knowledge  presavation  initiatives  being  pursued  at 
ONI  can  play  a  synergistic  role  within  the  three-pronged  knowledge  preservation  model.  Figure 
10  shows  how  ONI  ad  hoc  initiatives  relate  to  the  various  steps  of  the  three-pronged  model's 
select,  store,  and  actualize  processes.  As  related  above,  the  SME  group  can  help  identify 
knowledge  worth  preserving  by  identifying  analysts  with  expertise  and  can  help  to  establish  the 
future  relevance  of  analytical  knowledge.  The  key  indices  identified  in  the  knowledge  capture 
activities  of  the  video  and  case  study  stories  can  be  used  to  help  develop  the  cotporate  analysis 


Figure  10 

How  ONI  Ad  Hoc  Initiatives  Fit  Into  the  Three-Pronged  Model 
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standards.  The  interesting  stories  that  are  identified  can  be  used  to  develop  and  refine  both  the 
corporate  analysis  standards  and  the  analytical  tradecraft  process.  The  analytical  tradecraft 
process  can  point  toward  the  video  and  case  study  stories  as  specific  examples  of  the  process  in 
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action.  Finally,  the  “vignettes”  can  be  used  as  a  forum  for  learners  to  explain  the  case  studies 
they  have  developed.  Having  addressed  the  selection  of  knowledge  worth  preserving,  the 
discussion  now  turns  to  how  to  make  the  solution  practical  and  how  to  actualize  it. 

Characteristics  of  Practical  Solutions:  Actualize .  For  a  knowledge  preservation  solution 
to  be  seen  as  practical  by  analysts  and  senior  Navy  management,  both  groups  will  need  to 
perceive  the  value  provided  by  the  solution.  The  top  concerns  voiced  by  analysts  relate  to  ease 
of  use  and  how  much  using  the  solution  can  help  them  perform  their  jobs.143  Top  concerns 
voiced  by  ONI  management  include  a  desire  that  the  solution  be  “as  non-intrusive  as  possible” 
and  that  the  solution  be  effective  in  preserving  knowledge.144 

Top  Analyst’s  Concerns.  Concerns  analysts  have  over  ease  of  use  can  be  addressed  by 
beginning  small  with  pilot  projects  and  prototypes  that  involve  analysts  to  shape  the  “user 
interface”  and  resulting  system  behavior.  This  approach  requires  a  balance  to  obtain  enough 
analyst  involvement  to  ensure  ease  of  use  without  placing  excessive  demands  on  their  time.  For 
the  knowledge  preservation  solution  to  be  a  success,  some  knowledge  workers  must  be  dedicated 
full  time  to  the  project. 

Concerns  analysts  have  about  the  solution  being  able  to  help  them  do  their  jobs  will 
depend  on  the  quality  of  the  story-based  knowledge  captured  and  how  easily  the  end  products  are 
indexed  and  can  be  searched.  The  quality  of  the  knowledge  captured  will  depend,  in  part,  on  the 
ability  of  those  charged  with  selecting  knowledge  worth  preserving  to  select  the  right  knowledge 
and  by  the  skill  of  those  charged  with  capturing  the  knowledge  to  collect  the  contextual 
knowledge  and  metacognition  cues  that  help  learners  to  truly  understand  a  subject. 


143  Ibid. 

144  Ibid. 
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Top  Management  Concerns.  Management  concerns  about  the  solution  being  “as  non- 
intrusive  as  possible  hint  at  an  Artificial  Intelligence  (AI)  based  solution  that  could  “follow 
analysts  around  for  the  last  few  years  of  their  careers  to  capture  their  expertise.  Their  captured 
expertise  could  then  be  used  by  computer  support  systems  and  to  help  train  a  new  generation  of 
analysts.  Such  an  approach  would  appear  passive  and  non-intrusive  to  an  analyst  by  always 
staying  in  the  background  collecting  and  organizing  information. 

While  such  applications  may  appear  in  the  future,  they  are  not  available  today.  Products 

such  as  Tacit  Knowledge  System's  KnowledgeMail™  are  a  step  in  this  direction."'5  Here  is  how 
the  product  has  been  described: 

Tacit  has  developed  a  system  called  KnowledgeMail  that  automatically 
discovers  and  exchanges  expertise  under  the  secure  control  of  individual 
users,  and  integrates  profiling  and  expertise  search  capabilities  with  a 
road  range  of  desktop  applications.  The  system  automatically  and 
continuously  discovers  the  expertise  and  work  focus  of  everyone  in  the 
enterprise  and  then  brokers  connections  among  people  so  they  can  make 
timely,  informed  decisions  about  critical  and  complex  needs.146 

While  the  capabilities  of  KnowledgeMail™  are  no  doubt  impressive  and  to  some  degree 
unprecedented,  when  one  gets  down  to  the  product’s  essence,  it  is  no  more  than  a  very 
sophisticated  knowledge  map.  It  is  a  knowledge  map  that  helps  people  find  each  other  in  novel 
and  timely  ways  while  continuously  updating  the  map  to  reflect  changes  in  the  terrain. 
Unfortunately,  the  actual  knowledge  stills  resides  in  the  experts’  heads,  and  when  they  retire  it 
walks  off  the  map  with  them.  The  knowledge  map  then  points  to  an  empty  landmark.  As 
appealmg  as  this  type  of  Al-based  solution  appears  on  the  surface,  it  is  hard  to  conceive  of 
automated  software  technology  that  can  account  for  the  properties  of  the  “tree  of  talking” 

DeceSerG13d2001CIA  ^  Generation  formation  Technology  Tools,”  C4INews, 
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communication  model  described  in  the  previous  section,  i.e.,  software  capable  of  “tapping  into” 
the  exformation  from  both  learners  and  experts.  Based  on  the  science  of  learning,  such  software 
would  need  to  elicit  the  contextual  knowledge  and  metacognition  cues  that  people  use  to  learn 
with  understanding.  One  conclusion  developed  in  this  study  suggests  that  more  human 
interaction,  not  less,  makes  for  effective  knowledge  preservation  and  transfer. 

Management  concerns  about  the  solution  being  effective  at  preserving  knowledge  can  be 
addressed  by  following  a  process  that  balances  the  problem’s  knowledge,  learning,  assessment, 
and  cultural  dimensions.  The  knowledge  preservation  model  developed  in  this  study  report 
strives  to  achieve  this  balance.  Other  issues  of  practicality  include  the  solution’s  affordability, 
schedule,  and  cultural  acceptance.  The  following  discussion  explores  these  issues  and  presents 
some  ideas  for  how  to  manage  them. 

Affordability.  Affordability  is  one  of  those  terms  that  can  mean  different  things  to 

different  readers.  The  Department  of  Defense  uses  the  following  definition: 

Affordability  is  the  degree  to  which  the  life-cycle  cost  of  an  acquisition 
program  is  in  consonance  with  the  long-range  investment  and  force 
structure  plans  of  the  Department  of  Defense  or  individual  DoD 
Components.147 

The  long-range  investment  and  force  structure  plans  of  the  U.S.  Navy  must  include 
maintaining  and  bolstering  capabilities  for  Naval  intelligence  analysis.  The  shift  from  a  blue 
water  force  towards  a  forward-deployed  force  focused  on  the  littorals  and  the  proliferation  of 
high  technology  weapons  and  greater  numbers  of  potential  adversaries  dictate  that  the  demands 


147  Department  of  Defense,  Mandatory  Procedures  for  Major  Defense  Acquisition  Programs 
(MDAPS)  and  Major  Automated  Information  System  (MAIS)  Acquisition  Programs  (DoD 
5000.2-R,  April  5, 2002),  p.  70.  <http://dod5000.dau.mil/DOCS/Master.020405.Regulation.doc> 
[May  30, 2002], 
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on  Naval  intelligence  are  greater  today  than  during  the  Cold  War.  Consider  the  following 
excerpt  from  the  Navy  Strategic  Planning  Guidance  document: 


...  our  potential  adversaries  will  continue  to  pursue  area  denial  strategies 
over  the  next  15  to  20  years.  These  challenges  will  primarily  be  land- 
based  and  m  the  near-coastal  regions.  Some  potential  adversaries  will 
expand  their  denial  strategy  to  include  space  and  cyberspace  as  well  as 
adjacent  sea  and  air  space  to  provide  a  defense  in  depth.  Both  the 
sophistication  and  the  performance  of  weapons  will  increase 
substantially.  Platform  and  weapons  survivability  will  increase  through 
multi-spectral  signature  control  and  advanced  countermeasure  designs. 
Increasingly,  these  weapon  systems  and  platforms  will  be  supported  by 
more  sophisticated  sensor  systems.  This  requires  sophisticated  counter- 
targeting  systems  and  doctrine  to  enable  U.S.  and  allied  forces  to  defeat 
such  weapons.  There  will  be  significant  threats  to  forward  presence 
forces  from  defensive  mine  warfare;  massed  small  boat  attacks 
employing  man-portable  weapons;  advanced  air,  surface  and  submarine 
launched  cruise  missiles;  and  potential  for  chemical/biological 
weaponry.  Another  area  of  concern  is  the  availability  of  advanced  air 
independent  propulsion  systems  for  submarines. 148 


The  picture  painted  by  this  excerpt  indicates  that  an  investment  in  preserving  and 
transferring  knowledge  required  to  conduct  Naval  intelligence  analysis  is  warranted.  How  much 
should  one  expect  a  credible  knowledge  preservation  program  to  cost?  This  question  is 
answered  by  adopting  a  working  definition  of  affordability  and  performing  a  cost  analysis. 

A  working  definition  of  affordability  for  this  study  report  recognizes  the  total  ownership 
cost  aspects  of  life-cycle  costs,  but  simplifies  the  scope  to  consider  mainly  recurring  and  non¬ 
recurring  program  costs.  Non-recumng  costs  refer  to  the  costs  associated  with  acquisition  of 
hardware,  software,  and  the  technical  support  to  get  a  solution  up  and  running  Recurring  costs 
refer  to  the  costs  to  use  and  maintain  the  solution. 


148  Chief  of  Naval  Operations,  Navy  Strategic  Planning  Guidance  with  Long  Range  Planning 
Objectives,  April  2000,  pp.  5-6.  °  ° 
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Cost  Analysis.  There  are  several  cost  analysis  methodologies  that  can  be  used  to  estimate 
the  costs  associated  with  putting  the  three-pronged  knowledge  preservation  model  into  action. 
The  method  followed  here  draws  on  the  analogy,  parametric,  and  rule-of-thumb  approaches  to 
cost  estimation.149  The  cost  analysis  starts  by  focuses  primarily  on  the  “over-the-wall”  digital 
video  archive  component  of  the  three-pronged  knowledge  preservation  model.  Then  costs  for 
the  learner-built  case  studies  and  traditional  mentoring  are  estimated. 

“Full  Blown”  Sandia-Like  Analogy-based  Cost  Estimate.  An  analogy  can  be  drawn 
between  the  published  figures  for  Sandia  National  Laboratory’s  knowledge  preservation  program 
and  the  costs  for  a  knowledge  preservation  program  at  ONI.  Annual  recurring  costs  for  Sandia’s 
program  are  estimated  to  be  $1.2  million  per  year  in  FY98  dollars.  This  figure  is  based  on  two 
sources.  First,  a  DOE  Inspector  General’s  report  that  lists  an  estimate  of  Sandia’s  “costs  to  date” 
(covering  FY94-FY98)  as  $4.5  million  which  works  out  to  about  $1.3  million  per  year.150 
Second,  a  Sandia  Program  Plan  listing  knowledge  preservation  project  budget  line  of  $1.2 
million  per  year  for  FY98  through  FY02  in  FY98  dollars.151  $1.2  million  in  FY98  dollars  is 
equivalent  to  $1.46  million  in  FY03  dollars  assuming  an  annual  inflation  rate  of  4%.  The  Sandia 
budget  also  includes  the  following  potentially  relevant  costs  in  FY98  dollars:152 


Search  &  retrieval  tool  $  1,250K 

Automated  data  conversion  $  750K 

Pilot  nonelectronic  data  conversion  $  500K 

Classified  infrastructure _ $  100K 

Total  $  2,600K 


149  For  more  detailed  explanation  of  cost  analysis  methodologies  and  host  of  other  cost  analysis- 
related  knowledge  see  the  Defense  Acquisition  University’s  web  site  at 
<http://center.dau.mil/Topical_Sessions_templates/FMl/leaming-script.html>  [May  29,  2002]. 

150  DOE,  Efforts  to  Preserve  the  Knowledge  Base,  p.  5. 

151  Sandia  National  Laboratory,  A  Sandia  Nuclear  Weapon  Knowledge  Management  Program 
Plan  for  FY 1998-2003  (SAND97-3083/1  UC-70,  February  1998),  p.  17. 

152  Ibid.,  p.  17. 
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When  converted  from  FY98  into  FY03  dollars,  using  an  annual  inflation  rate  of  4%, 
$2,600K  becomes  $3,1 63K.  The  Sandia  budget  does  not  include  the  cost  to  line  organizations 
for  implementing  the  program  since  it  is  assumed  to  be  included  in  the  cost  of  “doing  business” 
and  the  budget  does  not  include  normal  Human  Resources  (HR)  and  Chief  Information  Officer 
(CIO)  costs  since  these  activities  are  assumed  to  be  funded  separately.153  The  Sandia  knowledge 
preservation  program  has  a  heavy  emphasis  on  video  with  over  2,000  hours  of  digital  video 
indexed  for  on-hne  access.154  Sandia’s  knowledge  preservation  program  also  includes  an  active 
document  archiving  component.  As  stated  previously,  although  there  are  tangible  benefits  from 
the  archiving  of  documents,  these  activities  were  not  included  as  an  integral  part  of  the 
knowledge  preservation  model  developed  for  this  study  report  because  the  archiving  of 
documents  is  not  seen  as  essential  for  the  transfer  of  expertise-related  knowledge  from  one 
analyst’s  head  to  another. 

Nevertheless,  a  raw  comparison  between  Sandia’s  experience  and  a  potential  knowledge 
preservation  solution  at  ONI  provides  an  idea  of  what  a  “full  blown”  knowledge  preservation 
program  could  cost.  Assuming  ONI’s  knowledge  preservation  program  “mirrors”  Sandia’s 
allows  their  budget  figures  to  be  converted  to  FY03  dollars  and  used  directly  to  estimate  the  cost 
of  a  “full  blown”  Sandia-like  program  at  ONI.  These  numbers,  in  FY03  dollars,  are  $1.46 
million  in  recurring  costs  and  $3,163  million  in  non-recurring  costs.  The  non-recurring  costs 

could  be  spread  over  several  years  as  the  program  ramps  up  while  the  recurring  costs  occur  every 
year. 


153  Ibid.,  p.  18. 

Dan  Caterimcchia,  Preserving  Knowledge,”  Federal  Computer  Week,  August  13  2001 

<http://www.fcw.eom/fcw/articles/2001/0813/mgt-doe-08-13-01.asp>  [May  30, 2002], 
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“Scaled  Down”  Sandia-Like  Analogy-based  Cost  Estimate.  Costs  for  a  “scaled  down” 
Sandia-like  solution  can  be  estimated  using  the  analogy  method.  In  this  case  it  is  assumed  that 
ONI  will  not  invest  in  document  archiving  activities  as  part  of  a  knowledge  preservation 
program.  The  Sandia  budget  figures  can  then  be  adjusted  by  halving  the  recurring  costs  to 
$730K.  The  rationale  for  this  change  stems  from  the  division  of  costs  between  video  and 
document  knowledge  preservation  activities.  This  division  is  not  specified  in  the  Sandia  budget, 
but  since  both  activities  are  known  to  be  sighificant  and  labor  intensive  endeavors,  a  logical  first 
order  approximation  is  to  divide  the  recurring  cost  figure  equally.  An  estimate  of  the  non¬ 
recurring  costs  in  this  case  can  be  made  by  using  only  the  Sandia  budget  figure  for  the  “search 
and  access  tool”  line.  The  rationale  for  this  change  is  that  the  “data  conversion”  lines  are 
assumed  to  be  related  to  document  archiving  efforts  and  ONI  can  leverage  Sandia’ s  investment 
in  developing  a  “classified  infrastructure.”  These  assumptions  result  in  an  adjusted  non¬ 
recurring  cost  of  $1,521K  in  FY03  dollars. 

Digital  Video  Pilot  Solution  Parametric-based  Cost  Estimate.  Costs  for  a  “pilot” 
knowledge  preservation  solution  using  a  parametric  method  can  be  made  using  the  following 
assumptions.  Again  it  is  assumed  that  ONI  will  not  invest  in  document  archiving  activities  as 
part  of  their  knowledge  preservation  program.  A  pilot  installation  of  Convera’s  Screening  Room 
digital  video  archiving  and  retrieval  software  will  be  acquired  along  with  video  cameras  and 
editing  equipment.  Full  time  personnel  dedicated  to  the  project  include  a  Knowledge  Manager 
and  a  Knowledge  Transfer  Engineer.155  Labor  costs  for  these  personnel  are  estimated  using  the 

155  For  descriptions  of  these  job  roles  see  Department  of  the  Navy,  Civilian  Career  Path  Guide 
for  the  Management  of  Technology,  Information,  and  Knowledge,  Vol.  II,  Career  Areas, 
Knowledge  Management,  March  2001,  pp.  KM-1  to  KM-3.  <http://www.don- 
imit.navy.mil/contentDownload.asp?theID=04272001  GIA4932846>  [May  30,2002] . 
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direct  labor  rates  for  Space  and  Naval  Warfare  System  Center,  Charleston  civil  service 
employees.156  The  Knowledge  Manager  is  assumed  to  be  a  DP04  (GS-14)  worker  with  a  FY03 
direct  man  year  rate  of  $176K  (calculated  from  the  FY02  labor  rate  accelerated  by  4%  for 
inflation).  Similarly,  the  Knowledge  Transfer  Engineer  is  assumed  to  be  a  DT03  (GS-12)  worker 
with  a  FY03  direct  labor  man  year  rate  of  $134K.  The  annual  recurring  cost  is  obtained  by 
adding  these  man  year  figures  together  plus  another  $50K  to  cover  material,  training,  travel,  and 
miscellaneous  expenses.  This  results  in  a  recurring  cost  of  $360K  in  FY03  dollars.  No  per  diem 
or  relocation  expenses  are  included  in  this  estimate.  The  rationale  for  these  numbers  is  that 
although  contractors  or  in-house  ONI  personnel  may  be  assigned  to  these  job  roles  for  the 
project,  their  labor  rates  should  be  no  greater.  The  non-recurring  cost  is  obtained  by  adding  a 
$100K  for  a  pilot  Convera  Screening  Room  suite  to  $25K  for  video  cameras  and  editing 
equipment.  This  Screening  Room  suite  consists  of  a  network-equipped  server,  workstation,  and 
a  five-seat  user  license.  The  cost  drivers  for  Screening  Room  are  the  seat  licenses  and  the  digital 
video  storage  hardware. 157  This  brings  the  total  non-recurring  costs  to  $125K  in  FY03  dollars. 

This  figures  are  summarized  below: 

Recurring  Costs 


Knowledge  Manager  (GS- 1 4)  $  1 76K 

Knowledge  Transfer  Engineer  (GS-12)  $  134K 

Materials,  Training,  &  Travel  $  50K 


Total  $  360K 


Space  and  Naval  Warfare  Systems  Center,  Charleston,  Fiscal  Year  2002  Navy  Working 
Capital  Fund  Rates  for  the  Space  and  Naval  Systems  Center,  Charleston,  SC 
(SPAWARSYSCENNOTE  7600, 10  October  2001). 

157  Alan  Smith,  telephone  conversation  with  author,  30  April  2002. 
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Non-Recurring  Costs 


Convera  Screening  Room  Suite  $  100K 

Video  Camera  &  Editing  Equipment _ $  25K _ 

Total  $  125K 

Digital  Video  Pilot  Solution  Rule-of-Thumb-based  Cost  Estimate.  Cost  for  a  “pilot” 
knowledge  preservation  solution  using  a  rule-of-thumb  method  can  be  made  using  Davenport 
and  Prusak’s  “331/3  rule.”158  The  331/3  rule  states  that  to  be  successful,  a  knowledge  management 
project  should  only  dedicate  one  third  of  its  time,  effort,  and  money  towards  technology.  This 
rule  can  be  applied  in  this  case  by  fixing  the  technology  costs  at  the  $125K  non-recurring  figure 
used  in  the  parametric  estimate  above.  This  fixes  the  recurring  costs  or  the  “non-technology” 
costs  at  $250K.  Both  figures  are  expressed  in  FY03  dollars. 

The  parametric  and  rule-of-thumb  estimates  above  have  centered  around  the  “over-the- 
wall”  digital  video  archive  component  of  a  three-pronged  knowledge  preservation  solution  for 
ONI.  Parametric  cost  estimates  for  the  “hand-off’  learner-built  case  studies  and  the  “traditional 
mentoring”  can  also  be  calculated. 

Learner-Build  Case  Study  Parametric-based  Cost  Estimate.  For  the  learner-built  case 
studies,  the  Retrieval  Ware  part  of  Convera’s  Screening  Room  software  can  be  used  to  put  the 
case  studies  on-line.  Although  the  case  studies  could  be  created  strictly  by  the  learner-retiring 
analyst  team  without  outside  assistance,  professional  help  with  interview  techniques,  knowledge 
elicitation,  and  on-line  conversion  should  be  included,  at  least  initially,  to  help  ensure  project 
success.  Therefore,  to  capture  these  costs  the  estimate  assumes  one  half  man-year  of  a  GS-14 
Knowledge  Manager.  When  these  labor  costs  are  combined  with  $15K  to  cover  materials, 

158  Davenport  and  Prusak,  Working  Knowledge, 173. 
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training,  travel,  and  miscellaneous  expenses  a  recurring  cost  of  $103K  in  FY03  dollars  is 
obtained.  The  non-recurring  costs  are  assumed  to  be  nil  since  the  Convera  Screening  Room  suite 
cost  is  captured  in  the  digital  video  archiving  estimates.  These  figures  are  summarized  below. 

Recurring  Costs 

Knowledge  Manager  (GS-14)  half  time  $  88K 

Materials,  Training,  &  Travel _ $  15K _ 

Total  $  103K 

Traditional  Mentoring  Parametric-based  Cost  Estimate.  For  traditional  mentoring, 
the  bulk  of  the  activity  occurs  between  the  mentor  and  the  mentee.  Therefore,  the  costs 
associated  with  traditional  mentoring  are  almost  entirely  buried  in  the  costs  of  conducting  normal 
business.  That  said,  the  traditional  mentoring  program  at  ONI  could  benefit  from  a  more 
formalized  structure,  from  counseling  and  training  of  participants,  and  from  active  efforts  to 
broker  mentoring  pairs.  One  option  is  to  redirect  existing  HR  personnel  to  provide  these 
services.  The  traditional  mentoring  component  of  a  knowledge  preservation  program  will  not 
require  additional  funding  if  improvements  to  the  mentoring  efforts  are  not  made,  or  if  existing 
HR  personnel  are  redirected  to  make  the  improvements.  However,  since  improvements  would 
be  beneficial  and  HR  personnel  are  undoubtedly  facing  their  own  challenges,  it  is  proposed  to 
dedicate  one  half  a  man-year  of  a  GS-14  Knowledge  Manager  to  the  traditional  mentoring 
component  of  a  knowledge  preservation  program.  This  would  allow  one  knowledge  worker  to 
split  her  time  between  leamer-built  case  studies  and  traditional  mentoring.  Using  the  same  logic 
as  above,  when  this  labor  cost  is  combined  with  $15K  to  cover  materials,  training,  travel,  and 
miscellaneous  expenses  a  recurring  cost  of  $103K  in  FY03  dollars  is  obtained.  Again,  the  non¬ 
recurring  costs  are  assumed  to  be  nil  since  no  equipment  is  required  for  traditional  mentoring. 
These  figures  (FY03  dollars)  are  summarized  below. 
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Recurring  Costs 


Knowledge  Manager  (GS-14)  half  time  $  88K 
Materials,  Training,  &  Travel _ $  15K 

Total  $  103K 


Cost  Analysis  Summary.  The  results  of  this  cost  analysis  are  summarized  in  Table  3. 
The  actual  costs  for  a  knowledge  preservation  program  at  ONI  probably  lie  somewhere  between 
the  extremes  listed  in  table  3.  A  “full  blown”  Sandia-like  solution  may  be  overkill  for  ONI’s 
needs  while  the  parametric  estimates  may  omit  some  important  costs.  The  value  of  this  cost 
analysis  is  not  in  the  absolute  accuracy  of  its  estimates,  but  by  giving  decision  makers  a  “feel” 
for  the  magnitudes  of  the  costs  involved  with  fielding  a  knowledge  preservation  program. 

Like  the  Sandia  budget,  the  cost  analysis  presented  here  does  not  capture  the  costs  for  the 
analysts’  time,  normal  HR  support,  and  Information  Technology  (IT)  network  support  because 
these  costs  are  assumed  to  be  accounted  for  elsewhere.  If  adequate  funding  cannot  be  obtained 
to  fund  a  pilot  solution  like  that  described  above,  it  may  still  be  possible  to  pursue  some 


Table  3 


Results  of  Cost  Analysis  for  ONI  Knowledge  Preservation  Solutions  using  Analogy, 
Parametric,  and  Rule  of  Thumb  Methods  (FY03  dollars). 


Cost 

Category 

“Full  Blown” 
Sandia-Like 
Solution 
(Analogy) 

“Scaled” 

Sandia-Like 

Solution 

(Analogy) 

Digital  Video 
Pilot  Solution 
(Parametric) 

Digital  Video 
Pilot  Solution 
(Rule-of- 
Thumb) 

Learner-Built 
Case  Studies 
(Parametric) 

Traditional 

Mentoring 

(Parametric) 

• 

Recurring 

$  1 ,460K 

$  730K 

$  360K 

$  250K 

$  103K 

$  103K 

Non¬ 

recurring 

$  3.163K 

$  1.521K 

$  125K 

$  125K 

$  OK 

$  OK 

Total 

$  4.623K 

$  2.251K 

$485K 

$  375K 

$103K 

$  103K 

knowledge  preservation  activities  including  videotaping  of  storytelling-interview  sessions. 
Critical  expertise-related  knowledge  could  still  be  captured,  but  its  effective  use  and  transfer 
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would  be  limited  until  the  supporting  technology  could  be  fielded  and  post-production  activities 
could  be  performed. 

The  results  in  table  3  can  be  interpreted  by  defining  a  range  of  knowledge  preservation 
solutions.  These  figures  range  from  $478K  to  $69 IK  in  FY03  dollars.  The  low  end  is  calculated 
by  adding  the  rule-of-thumb  estimate  of  S375K  for  a  digital  video  to  the  parametric  estimate  of 
S103K  for  leamer-built  case  studies.  This  low  end  estimate  assumes  no  costs  for  traditional 
mentoring.  The  high  end  is  calculated  by  adding  the  parametric  estimate  of  $485K  to  the 
parametric  estimates  of  $103K  for  both  the  leamer-built  case  studies  and  traditional  mentoring. 

When  compared  with  the  costs  for  Sandia’s  knowledge  preservation  program,  these  costs 
seem  affordable.  However,  costs  should  also  be  judged  against  the  potential  benefits  from 
reducing  the  6-8  years  it  takes  to  develop  a  top  notch  Naval  intelligence  analyst  and  increasing 
the  overall  quality  of  intelligence  analysis. 

Schedule.  The  schedule  for  implementing  a  knowledge  preservation  program  at  ONI 
depends  on  the  resources  that  can  be  made  available  and  the  commitment  of  the  organization  to 
preserving  knowledge.  Some  recent  data  suggests  that  some  federal  workers  are  retiring  later 
than  first  projected  (at  the  Social  Security  Administration,  for  example),  but  several  of  the  senior 
analysts  this  author  interviewed  stated  that  they  had  firm  plans  to  retire  in  2003  and  2004.159’160 
The  sooner  a  knowledge  preservation  plan  can  be  put  into  action,  the  sooner  expertise-related 
knowledge  can  start  being  preserved  in  a  systematic  process.  A  pilot  project  for  digital  video 
could  be  started  in  FY03  assuming  project  plans  can  be  completed  and  funding  can  be  obtained. 
The  leamer-built  case  studies  could  begin  before  FY03  assuming  the  learners  and  retiring  experts 

159  “Reality  Check,”  Government  Executive,  May  2002,  p.  28. 

160  ONI  Representatives,  Interview  by  author,  Suitland,  MD,  3  April  2002. 
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can  be  identified.  Traditional  mentoring  is  already  taking  place  at  ONI.  However,  mentoring 
could  be  more  formalized  and  more  widely  encouraged. 

Cultural  Acceptance.  To  understand  the  impact  that  culture  and  cultural  acceptance  have 
on  the  success  of  a  knowledge  preservation  program,  it  helps  to  start  by  considering  what  is 
meant  by  culture. 

Culture  is  the  combination  of  shared  history,  expectations,  unwritten 
rules,  and  social  mores  that  affects  the  behavior  of  everyone,  from 
managers  to  mailroom  clerks.  It’s  the  set  of  underlying  beliefs  that, 
while  never  exactly  articulated,  are  always  there  to  color  the  perception 
of  actions  and  communications.161 

“Culture  is  no  more  and  no  less  than  shared  expectations,”  writes  Roger  Schank.162  If  culture  is 
bound  up  in  people’s  expectations,  what  expectations  act  as  potential  barriers  to  a  knowledge 
preservation  program?  The  most  obvious  barrier  occurs  when  people  have  an  information¬ 
hoarding  versus  an  information-sharing  attitude.  Another  barrier  is  erected  when  people  have  a 
sense  of  not  being  involved  in  the  decision  making  process.  Finally,  another  barrier  appears 
when  people  are  not  motivated  to  participate  in  the  process  due  to  a  perceived  lack  of  value, 
difficulty  of  use,  or  an  absence  of  long-term  incentives.  These  expectational  barriers  appeared  in 
the  knowledge  management  lessons  discussed  in  section  IV,  where  the  guidelines  to  adopt  a 
knowledge-oriented  culture  and  to  use  nontrivial  motivational  aids  were  discussed. 

Although  organizational  cultures  can  be  resistant  to  change,  workers’  expectations  can  be 
influenced  by  understanding  and  paying  attention  to  the  issues  that  drive  their  behavior.  For 
example,  a  reluctance  to  share  information  can  be  caused  by  a  worker’s  fear  of  job  loss  or  loss  of 

161  Carla  O’Dell  and  C.  Jackson  Grayson,  Jr.,  If  We  Only  Knew  What  We  Know:  The  Transfer  of 
Internal  Knowledge  and  Best  Practice  (New  York,  NY:  The  Free  Press,  1998),  p.  71. 

162  Roger  C.  Schank,  The  Connoisseur’s  Guide  to  the  Mind:  How  We  Think,  How  We  Learn,  and 
What  It  Means  to  be  Intelligent  (New  York,  NY:  Summit  Books  (Simon  &  Schuster),  1991),  p. 
42. 


101 


prestige.  This  issue  can  be  addressed  by  rewarding  workers  who  share  knowledge  and  by 
fostering  interdependence  and  taking  steps  to  build  trust  between  workers.  Getting  analysts 
involved  in  the  planning  stages  of  a  knowledge  preservation  program  and  soliciting  and  acting  on 
their  feedback  can  lead  workers  to  develop  a  sense  of  ownership  in  the  process. 

The  issue  of  motivating  analysts  to  participate  in  knowledge  preservation  activities  can  be 

addressed  in  several  ways.  For  retiring  analysts,  emphasizing  knowledge  preservation  as  an 

opportunity  for  them  to  leave  a  legacy  behind  can  provide  a  strong  incentive  for  them  to 

participate.  Most  of  these  analysts  are  eager  to  have  their  knowledge  preserved,  but  they 

currently  do  not  have  the  resources  and  the  infrastructure  at  their  disposal  to  easily  make  it 

happen.  For  junior  analysts,  the  greatest  motivation  will  come  when  they  realize  that  their 

participation  is  helping  them  to  do  their  jobs  better.  For  both  groups  of  analysts,  a  key  spark  that 

can  ignite  a  self-sustaining  knowledge  preservation  program  is  to  make  the  process  “seem  like 

fun.”  One  way  to  achieve  this  is  to  treat  knowledge  transfer  exchanges,  regardless  of  the  mode, 

as  internal  conversations.  Drawing  from  the  ideas  presented  in  section  V: 

If  we  view  conversations  from  without,  as  information  being  exchanged, 
they  are  not  especially  rich.  But  if  we  see  them  from  within,  as 
exformation,  they  can  be  tremendous  fun.  If  you  do  not  know  the 
context,  they  can  be  dull.  It  is  very  boring  to  listen  to  people  talking 
about  someone  you  do  not  know.  Such  conversations  tell  you  very  little. 

But  it  is  fun  to  talk  about  people  you  do  know,  whether  personally  or  as 
public  figures.163 


The  secret  to  making  knowledge  preservation  seem  like  fun  is  to  get  people  to  really  know  each 
other  they  need  to  develop  meaningful  relationships.  How  might  this  be  accomplished  when  a 
newer  analyst’s  only  exposure  to  a  retired  analyst  is  through  digital  video  clips  and  archived 
documents?  One  way  is  to  record  some  interview  sessions  that  capture  the  personality  of  the 


163  Norretranders,  The  User  Illusion,  p.  96. 
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retiring  analyst  in  addition  to  the  structured  storytelling  sessions  that  are  intended  to  capture 
expertise-related  knowledge.  Such  interview  sessions  could  include  experiences  from  an 
analyst’s  youth  and  school  days  that  shaped  their  career  or  inspired  them.  Anecdotes  about  an 
analyst’s  hobbies,  favorite  foods,  and  travel  could  be  included  as  well. 

Although  I  never  met  the  physicist  Richard  Feynman,  I  feel  like  I  know  him  by  reading 
about  his  exploits  where  he  fixes  radios  by  thinking,  cracks  safes  at  Los  Alamos,  and  plays 
bongos  in  Brazil.164  Providing  a  sort  of  “biography  channel”  with  similar  content  in  addition  to 
the  expertise-related  knowledge  could  also  provide  newer  analysts  with  the  same  sort  of  window 
on  the  personalities  of  their  retired  predecessors. 

This  biographical  background  knowledge  can  be  used  to  introduce  newer  analysts  to  the 
characters  of  retired  analysts  from  whom  they  will  be  learning.  This  can  serve  to  make  the 
learning  experience  richer,  more  interesting,  and  therefore,  more  fun. 

Knowledge  Transfer  and  Competitive  Positions.  One  potential  barrier  to  knowledge 
transfer  activities  is  the  perception  that  special  attention  given  to  analysts,  who  have  mentoring 
relationships  or  are  building  case  studies  with  their  senior  counterparts,  conflicts  with  civilian 
personnel  policies  on  advertised  competitive  positions.  One  senior  analyst  interviewed  by  the 
author  explained  that  this  issue  was  raised  in  a  Subject  Matter  Experts  (SME)  group  meeting  at 
ONI.165 

One  solution  to  this  problem  is  to  decouple  knowledge  transfer  activities  from  the  SME 
positions.  While  ad  hoc  activities,  outside  the  auspices  of  a  formal  program,  could  possibly  be 

164  Edward  Hutchings  (Ed.),  Ralph  Leighton,  and  Richard  P.  Feynman,  'Surely  You  're  Joking, 

Mr.  Feynman!’:  Adventures  of  a  Curious  Character  (New  York,  NY:  W.  W.  Norton  & 

Company,  1997). 

165  ONI  Representatives,  Interview  by  author,  Suitland,  MD,  3  April  2002. 
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construed  as  favoritism,  incorporating  these  activities  as  part  of  a  career  development  program 
for  analysts  removes  this  concern.  Identifying  analytical  competencies  and  career  plans  in 

writing  are  also  in  consonance  with  the  analytical  tradecraft  process  initiatives  discussed  earlier 
(see  pages  87-89). 

An  approach  similar  to  the  one  used  in  the  Civilian  Career  Path  Guide  could  be  adopted 
to  document  career  foundational  competencies”  and  provide  assessment  tools  for  Naval 
intelligence  analysts.166  Elements  of  this  approach  include: 

•  Job  roles  and  competency  lists167 

•  Assessment  matrices168 

•  Career  Progression  Plans  (CPP)  or  Individual  Development  Plans  (IDP)169 

Embedding  knowledge  transfer  and  preservation  activities  in  a  formal  career  development 

program  for  analysts  has  the  advantages  of  setting  written  goals,  providing  an  individual 

assessment  mechanism,  and  documenting  progress.  The  Civilian  Career  Path  Guide  provides 

the  following  advice  with  regard  to  career  planning  activities  and  competitive  positions: 

Keep  in  mind  that  career  planning  activities  do  not  guarantee  promotion 
or  advancement,  but  do  enhance  an  employee’s  ability  to  advance  or  be 
promoted.170 

Summary.  In  this  section,  specific  answers  have  been  developed  and  presented  to  the 
research  questions  raised  earlier  in  the  report.  The  nature  of  expertise  in  conducting  Naval 
intelligence  was  discussed  along  with  ideas  on  how  to  identify  and  prioritize  expertise-related 
knowledge  worth  preserving.  Interview  techniques  for  eliciting  knowledge  and  ideas  for 
weaving  stories  and  narratives  around  critical  indices  and  cues  were  also  presented.  The  benefits 

166  Department  of  the  Navy,  Civilian  Career  Path  Guide,  pp.  35-36. 

Ibid.,  pp.  KM-1  through  KM-5. 

168  Ibid.,  pp.  35-36. 

169  Ibid.,  p.  47. 
m  Ibid.,  p.  47. 
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of  a  systematic  process  for  archiving  documents,  files,  and  records  were  discussed  along  with 
how  a  document  archiving  process  can  complement  a  knowledge  preservation  program.  The 
value  of  adopting  a  process  orientation  towards  intelligence  analysis  was  also  discussed. 

The  ad  hoc  knowledge  preservation  initiatives  being  taken  at  ONI  were  discussed  and 
ideas  for  incorporating  these  initiatives  into  the  three-pronged  knowledge  preservation  model 
were  presented.  Finally,  the  characteristics  of  what  it  takes  for  a  practical  knowledge 
preservation  program  was  addressed  by  considering  top  concerns  voiced  by  both  analysts  and 
ONI  management.  This  discussion  was  followed  by  considering  the  issues  of  affordability, 
schedule,  and  cultural  acceptance. 

Affordability  was  addressed  by  adopting  a  simplified  working  definition  and  performing 
a  cost  analysis  using  analogies  drawn  with  Sandia  National  Laboratory’s  knowledge  preservation 
program.  Additional  cost  estimates  using  parametric  and  rule-of-thumb  cost  estimate  methods 
were  also  presented.  The  results  from  this  cost  analysis  were  then  interpreted  and  set  in  a  larger 
context  by  defining  a  range  of  options. 

Schedule  issues  were  addressed  by  discussing  when  various  components  of  a  knowledge 
preservation  program  could  reasonably  be  put  into  action.  Given  a  high  priority,  all  components 
could  be  in  place  in  FY03. 

Cultural  issues  were  addressed  by  discussion  how  culture  translates  into  shared 
expectations  and  what  can  be  done  to  remove  cultural  barriers  to  a  knowledge  preservation 
program  and  make  it  seem  like  fun.  In  the  next  section  areas  for  future  research  are  identified 
and  concluding  remarks  are  presented. 
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SECTION  VII 


CONCLUDING  REMARKS 

Areas  for  Future  Study.  There  are  a  number  of  important  issues  which  are  outside  the 
scope  of  the  work  presented  in  this  study  report.  These  issues  are  ripe  for  future  study  and 
answers  to  the  questions  they  raise  could  greatly  assist  knowledge  management  programs  at  DoD 
in  general,  and  ONI  in  particular.  These  issues  include: 

•  Assessment  The  best  ways  to  assess  and  measure  the  effectiveness  of  knowledge 
preservation  efforts  are  open  questions.  Is  there  a  common  method  to  assess  learning  and 
the  effectiveness  of  expertise-related  knowledge  capture  using  a  digital  video  archive, 
leamer-built  case  studies,  and  traditional  mentoring  approaches? 

•  Document  Archiving.  The  archiving  of  documents,  files,  and  records  was  judged  to  be 
beneficial,  but  not  essential,  for  the  transfer  of  knowledge  from  one  analyst’s  head  to 
another.  The  value  an  electronic  document  archive  can  add  to  the  knowledge 
preservation  process  should  be  studied. 

•  Use  of  Al-based  Tools.  Although  Artificial  Intelligence  based  tools  are  not  yet  capable 
of  true  knowledge  preservation  and  transfer  between  people,  products  such  as  Tacit 
Knowledge  System’s  KnowledgeMail  offer  new  possibilities  for  identifying  and  selecting 
knowledge  worth  preserving.  These  tools  should  be  studied  and  used  if  they  can  be 
shown  to  help  select  knowledge  worth  preserving. 

•  Enterprise-Wide  Knowledge  Management.  How  should  knowledge  preservation  be 
integrated  into  an  organization’s  overall  enterprise-wide  knowledge  management 

program?  Opportunities  for  leveraging  resources  and  efforts  should  be  studied  and  acted 
on. 
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•  Advanced  Automated  Analytic  Tools.  The  expertise-related  knowledge  captured  in  a 
knowledge  preservation  program  represents  a  resource  that  can  be  tapped  to  create 
advanced  automated  analytic  tools.  These  opportunities  should  be  studied  and  efforts 
made  to  fully  exploit  this  source  of  knowledge  for  the  development  of  advanced 
automated  analytic  tools. 

•  Phased  Retirement.  Congress  is  considering  legislation  that  would  remove  financial 
barriers  to  allow  “phased  retirement”  where  retired  employees  can  work  part-time  and 
still  receive  their  full  annuities.171  Initiatives  such  as  phased  retirement,  part-time 
employment,  or  contract  employment  to  make  selected  retired  analysts  with  critical 
knowledge  available  for  knowledge  capture  activities  should  be  studied. 

Conclusion.  Despite  the  attention  now  being  given  to  the  problems  of  an  aging 
workforce,  preserving  the  expertise-related  knowledge  of  seasoned  workers  before  they  retire 
remains  a  daunting  task.  The  knowledge  itself  is  difficult  to  capture  and  store  in  a  form  that  can 
easily  be  absorbed  and  used  by  others.  The  time  and  resources  that  can  be  dedicated  toward  this 
problem  are  in  limited  supply.  The  argument  presented  in  this  study  report  is  that  these  obstacles 
to  knowledge  preservation  can  be  overcome  by  following  a  planned  approach  that  makes  the  best 
use  of  the  time  available,  uses  stories  and  narrative  to  convey  knowledge,  and  builds  on  the 
science  of  learning. 

In  this  study  report,  questions  about  the  nature  of  expertise,  the  transfer  of  expertise- 
related  knowledge,  and  the  practicality  of  potential  solutions  for  a  knowledge  preservation 
program  at  the  Office  of  Naval  intelligence  were  examined.  The  answers  to  these  questions  led 

171  ERISA  Advisory  Council,  “The  Working  Group  Report  on  Phased  Retirement,”  November 
14, 2000.  <http://www.dol.gov/dol/pwba/public/adcoun/phasedrl.htm>  [May  10, 2002]. 
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to  the  development  of  a  three-pronged  model  for  knowledge  preservation  and  ideas  for  putting 
the  model  into  action  that  leverage  ad  hoc  ONI  knowledge  preservation  initiatives.  The  issue  of 
affordability  was  addressed  through  a  cost  analysis  that  provides  some  order  of  magnitude 
estimates  for  the  cost  of  a  knowledge  preservation  program  at  ONI.  Barriers  to  knowledge 

preservation  and  transfer  including  cultural  acceptance  and  civilian  personnel  policies  were 
considered. 

While  important  issues  remain  to  be  studied,  enough  is  known  to  begin  attacking  the 
knowledge  preservation  problem  at  ONI.  The  theory,  lessons,  model,  and  analysis  contained  in 
this  study  report  provide  a  basis  for  building  a  viable  knowledge  preservation  program. 
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